-- Logs begin at Mon 2022-08-01 14:25:09 UTC, end at Mon 2022-08-01 15:40:36 UTC. -- Aug 01 14:59:10.737996 dv-cinder-843277-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 01 14:59:10.749179 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 01 14:59:10.751251 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 1 14:59:10 2022] *** Aug 01 14:59:10.751251 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 01 14:59:10.751251 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 01 14:59:10.751251 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: nodename: dv-cinder-843277-7-iscsi Aug 01 14:59:10.751251 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: machine: x86_64 Aug 01 14:59:10.751251 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: clock source: unix Aug 01 14:59:10.751251 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: pcre jit disabled Aug 01 14:59:10.751251 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: detected number of CPU cores: 4 Aug 01 14:59:10.751251 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: current working directory: / Aug 01 14:59:10.751251 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: detected binary path: /usr/bin/uwsgi-core Aug 01 14:59:10.751886 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: your processes number limit is 47680 Aug 01 14:59:10.751886 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: your memory page size is 4096 bytes Aug 01 14:59:10.751886 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: detected max file descriptor number: 2048 Aug 01 14:59:10.751886 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: lock engine: pthread robust mutexes Aug 01 14:59:10.751886 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: thunder lock: enabled Aug 01 14:59:10.751886 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 01 14:59:10.751886 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 01 14:59:10.790689 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: Python main interpreter initialized at 0x5558dbc0f9c0 Aug 01 14:59:10.790689 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: python threads support enabled Aug 01 14:59:10.790689 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: your server socket listen backlog is limited to 100 connections Aug 01 14:59:10.790943 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: your mercy for graceful operations on workers is 90 seconds Aug 01 14:59:10.791018 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: mapped 403077 bytes (393 KB) for 2 cores Aug 01 14:59:10.791113 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: *** Operational MODE: preforking *** Aug 01 14:59:10.791384 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: *** uWSGI is running in multiple interpreter mode *** Aug 01 14:59:10.791384 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: spawned uWSGI master process (pid: 80348) Aug 01 14:59:10.791572 dv-cinder-843277-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 01 14:59:10.794020 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: spawned uWSGI worker 1 (pid: 80358, cores: 1) Aug 01 14:59:10.794020 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: spawned uWSGI worker 2 (pid: 80359, cores: 1) Aug 01 14:59:10.794020 dv-cinder-843277-7-iscsi devstack@keystone.service[80348]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 01 14:59:16.407644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 14:59:16.407644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: from cryptography.utils import int_from_bytes Aug 01 14:59:16.408460 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 14:59:16.408460 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: from cryptography.utils import int_from_bytes Aug 01 14:59:16.410084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 14:59:16.410084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: from cryptography.utils import int_from_bytes Aug 01 14:59:16.410542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 14:59:16.410542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: from cryptography.utils import int_from_bytes Aug 01 14:59:16.458576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 14:59:16.458811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 14:59:16.458975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] command line args: [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 14:59:16.459256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 14:59:16.459435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ================================================================================ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 14:59:16.459700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] admin_token = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.459953 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config_dir = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.460038 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 14:59:16.460144 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.460222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 14:59:16.460329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] config_source = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.460413 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] command line args: [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 14:59:16.460496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] debug = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.460599 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 14:59:16.460700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.460961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ================================================================================ {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 14:59:16.461037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] admin_token = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.461102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.461247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.461333 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] config_dir = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.461434 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.461507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.461624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] config_source = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.461736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.461815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] debug = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.461974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.462054 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.462149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_config_append = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.462216 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.462386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.462509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.462599 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_dir = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.462782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.462871 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.463260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.463841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.464027 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.464122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] list_limit = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.464200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] log_config_append = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.464325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.464401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.464523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.464608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] log_dir = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.464680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.464753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] log_file = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.464862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.464993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.465115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.465200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.465399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.465492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.465646 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.465780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.465860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.465974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.466055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.466136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.466252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.466334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.466532 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.466615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.466802 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.466904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.467040 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.467160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.467261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.467496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.467635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.100/identity {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.467747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.467824 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] publish_errors = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.467993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.468077 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.468268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.468345 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.468449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.468700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.468776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.100/identity {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.468941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.469018 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] publish_errors = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.469172 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.469255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] standard_threads = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.469403 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.469490 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.469563 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.469637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.469811 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.469945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.470015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_journal = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.470094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.470166 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_json = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.470269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] standard_threads = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.470352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_stderr = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.470478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] use_syslog = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.470560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.470733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.470825 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.470926 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.471009 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.471092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] use_journal = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.471226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.471317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] use_json = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.471429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] use_stderr = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.471512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.471607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] use_syslog = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.471725 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.471839 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 14:59:16.471980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.472099 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.472227 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.472337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.472481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.472605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.472724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.472846 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.472977 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.473097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.473232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.473340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.473492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.473589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.473733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.473810 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.473988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.474071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.474230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.474306 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.474489 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.474572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.474766 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.474860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.475014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.475093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.475256 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.475332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.475505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.475585 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.475756 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.475833 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.475994 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.476070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.476249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.476326 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.476506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.476583 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.476755 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.476833 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.476994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.477078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.477232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.477308 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.477471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.477547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.477710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.477846 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.478024 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.478172 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.478305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.478442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.478570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.478716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.478837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.479025 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.479169 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.479308 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.479431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.479554 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.479679 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.479801 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.479927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.480061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.480194 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.480319 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.480448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.480571 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.480692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.480836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.480960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.481084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.481208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.481335 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.481456 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.481653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.481777 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.481899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.482025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.482151 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.482273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.482412 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.482533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.482681 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.482802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.482925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.483045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.483187 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.483309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.483433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.483564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.483692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.483811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.483945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.484066 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.484197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.484459 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.484538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.484605 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.484674 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.484734 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.484792 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.484864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.484925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.485018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.485090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.485151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.485209 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.485273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.485333 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.485391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.485459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.485525 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.485585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.485643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.485701 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.485827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.485910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.486098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.486765 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.487012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.487093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.487155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.487258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.487345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.487452 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.487531 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.487600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.487703 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.487781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.487881 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.488003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.488125 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.488250 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.488327 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.488388 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.488499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.488615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.488692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.488752 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.488851 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.488970 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.489058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.489160 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.489238 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.489344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.489458 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.489534 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.489640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.489763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.489889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.490022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.490136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.490268 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.490375 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.490478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.490608 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.490783 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.490888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.491092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.491229 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.491310 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.491465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.491542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.491644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.491723 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.491833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.491913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.492100 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.492238 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.492367 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.492491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.492697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.492830 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.492957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.493085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.493210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.493355 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.493491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.493941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.494087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.494219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.494346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.494474 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.494598 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.494823 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.494957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.495093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.495230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.495358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.495496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.495635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.495771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.495897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.496032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.496166 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.496292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.496420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.496554 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.496682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.496809 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.496943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.external = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.497071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.497217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.497352 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.497478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.497603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.497731 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.497855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] auth.external = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.498011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.password = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.498167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.498300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] auth.token = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.498423 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.498548 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.498646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.498729 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.498791 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] auth.password = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.498863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.499010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] auth.token = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.499144 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.499270 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.499396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.499520 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.499653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.499736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.499799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.499931 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.500138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.500223 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.500294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.500362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.500424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.500484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.500874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.500999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] credential.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.501136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.501261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.501391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.501513 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.501637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.501764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.501899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.501899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.502140 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.502264 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.502407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.502532 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.502697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.502834 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.502959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.503083 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.503206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.503327 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.503461 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.503587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.503723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.503849 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.503974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.504098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.504098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.504353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.504479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.504603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.504727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.504851 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.504984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.505109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.505233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.505358 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.505487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.505612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.505752 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.505887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.506011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.506135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.506259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.506383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.506521 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.506646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.506783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.506913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.507037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.507161 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.507298 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.507423 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.507557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.507640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.507765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.507963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.508091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.508214 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.508340 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.508424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.508498 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.508618 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.508735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.508813 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.508929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.509006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.509079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.509722 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.509808 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.509926 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.510004 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.510119 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.510196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.510320 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.510399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.510515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.510591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.510721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.510799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.510927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.511006 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.511123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.511123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.511313 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.511391 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.511514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.511593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.511708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.511785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.511902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.511902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.512034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.512097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.512217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.512295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.512410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.512487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.512611 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.512689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.512763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.512835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.512907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.513023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.513108 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.513189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.513262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.513335 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.513407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.513522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.513607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.513682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.513754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.513827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.513899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.514015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.514100 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.514175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.514247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.514319 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.514501 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.514582 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.514767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.514852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.515017 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.515100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.515268 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.515348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.515503 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.515585 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.515744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.515823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.515974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.516053 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.516216 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.516297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.516455 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.516538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.516689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.516769 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.516928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.517008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.517167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.517246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.517410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.517489 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.517653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.517733 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.517884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.517962 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.518117 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.518207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.518362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.518552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.518638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.518879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.519046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.519167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.519373 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.519478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.519626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.519864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.519944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.520096 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.520271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.520351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.520561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.520643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.520803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.520891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.521038 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.521118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.521301 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.521382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.521556 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.521643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.521809 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.521891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.522064 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.522201 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.522306 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.522458 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.522539 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.522720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.522806 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.522979 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.523059 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.523226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.523308 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.523469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.523549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.523712 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.523792 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.523956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.524037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.524204 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.524283 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.524453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.524531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.524697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.524778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.525527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.525620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.525770 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.525877 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.526056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.526191 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.526316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.526442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.526573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.526720 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.526856 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.526941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.527061 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.527186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.527315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.527444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.527593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.527723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.527852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.527980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.528108 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.528236 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.528372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.528498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.528576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.528649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.528722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.528793 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.528874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.529020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.529101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.529271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.529351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.529518 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.529608 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.529754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.529834 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.529993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.530072 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.530234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.530321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.530481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.530560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.530745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.530830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.530988 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.531151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.531230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.531303 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.531481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.531562 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.531718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.531806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.531973 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.532080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.532209 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.532290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.532451 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.532541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.532689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] role.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.532770 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.532925 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.533004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] role.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.533167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.533247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.533416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.533495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.533661 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.533741 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.533910 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.533990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.534155 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.534360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.534440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.534603 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.534717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.534893 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.535003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.535138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.535246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.535392 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.535602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.535691 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.535864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.535951 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.536121 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.536204 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.536372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.536456 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.536613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.536697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.536853 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.537118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.537230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.537297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.537358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.537522 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.537631 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.537809 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.538009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.538094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.538267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.538352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.538687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.538822 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.538822 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.539036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.539124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.539294 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.539378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.539535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.539642 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.539852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.540001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.540112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.540215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.542542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.542698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.542808 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.542907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.543006 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.543103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.543218 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.543320 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.543418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.543515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.543612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.543708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.543813 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.543913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.544021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.544119 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.544218 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.544825 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.544907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.544982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.545055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.545127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.545198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.545279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.545354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.545425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.545496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.545568 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.545640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.545723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.545804 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.545885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.545959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.546030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.546030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.546160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.546231 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.546311 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.546384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.546456 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.546527 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.546598 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.546725 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.546817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.546921 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.547023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.547099 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.547170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.547170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.547362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.547441 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.547515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.connection = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.547595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.547676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.547751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.547823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.547903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.547978 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.548050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.548122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.548193 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.548265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.548344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.548418 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.548490 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.548562 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.548681 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80358) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 14:59:16.548759 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.548759 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.548759 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.548958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.549169 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.549421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.549662 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.549911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.550143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.550385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.550624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 14:59:16.550814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80359) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 14:59:16.578207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.578435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.578656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.579762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.580002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.580222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.587292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.589584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.618138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.618369 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.619710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.619999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.674014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.676236 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.730294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.730515 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.732401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.732636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.797958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.798178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.798314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.798469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.798587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.798870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.798988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.799109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.799225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.799358 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.799471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.799471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.824027 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.824263 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.824380 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.824457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.824523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.824588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.824678 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.824762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.824895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.824967 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.825106 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.825176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.825318 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.825398 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.836903 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.837217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.850625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.850882 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.851058 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.851203 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.851348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.851348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.851602 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.851753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.851753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.852003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.852138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80359) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.852275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.852477 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.852789 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80358) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 14:59:16.869673 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.869881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 14:59:16.869971 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.870114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 14:59:16.870217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.870411 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 14:59:16.870493 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.870645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 14:59:16.870751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.870944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.871039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.871203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.871283 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.871460 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.871593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.871726 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.881199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.881481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.881587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.881751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.881843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.882027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.882115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.882383 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.889713 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.890003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.894558 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.894857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.894944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.895145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.895229 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.895425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.895511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.895706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.895789 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.895966 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.896058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.896237 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.896321 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.896509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.896591 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.896775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.896865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.897046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.897127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.897308 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.897389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.897575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.920556 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.920963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.922064 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.922343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.922499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.922815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.927154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.927375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.927503 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.927634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.927775 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.928026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.935108 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.935342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.935430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.935633 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.939966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.940187 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.940276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.940477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.940578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.940758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.940934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.941060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.941265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.941348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.941554 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.941637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.941836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.941917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.942111 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.942386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.958621 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.958927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.959106 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.959393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.962953 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.963640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.966409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.966730 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.966815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.967088 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.971197 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.971588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.974683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.974785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.977863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.978018 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.978207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.978320 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.978498 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.978606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.978822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.978933 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.979115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.979197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.979408 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.979500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.996301 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.996419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:16.996588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.996671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.996855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.996946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.997113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:16.997196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.004064 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.004250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.005609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.005726 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.005887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.005978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.008723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.008833 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.009005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.009089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.009895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.010019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.010179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.010355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.010472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.010646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.027855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.028311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.028602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.028882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.029162 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.029433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.029705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.030427 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.030771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.031097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.031355 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.031643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.031927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.032207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.047114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.048136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.050200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.051225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.053168 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.054206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.055799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.057436 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.057850 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.058146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.058452 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.059125 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.059428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.059712 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.064966 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.066260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.066567 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.067956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.069660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.069947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.070229 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.070500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.070928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.071225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.071506 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.072456 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.079267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.079607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.079984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.080685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.081107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.081469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.087562 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80359) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.087878 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.088418 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.088728 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.089014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.089312 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80358) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 14:59:17.089453 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.089577 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.089709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.089904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.090035 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.090184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.090314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.090486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.090609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.090846 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.090992 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.091198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.091327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.091451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.091577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.091771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.091916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80359) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.092042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.092373 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.092648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.092939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.093206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80358) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 14:59:17.128721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5558dbc0f9c0 pid: 80358 (default app) Aug 01 14:59:17.131605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5558dbc0f9c0 pid: 80359 (default app) Aug 01 14:59:17.132162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea5d60-c6e5-48a2-9881-6716c08f4a0e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:17.132453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea5d60-c6e5-48a2-9881-6716c08f4a0e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:17.132731 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea5d60-c6e5-48a2-9881-6716c08f4a0e None None] PATH_INFO: `/v3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:17.137094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1/1] 10.222.0.100 () {50 vars in 796 bytes} [Mon Aug 1 14:59:17 2022] GET /identity/v3/ => generated 256 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:24.886955 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5143a4-5eae-4294-a994-1d160cacc615 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:24.887542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5143a4-5eae-4294-a994-1d160cacc615 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:24.887542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5143a4-5eae-4294-a994-1d160cacc615 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:24.891973 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1/2] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:24 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:24.897468 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eaff8-f563-4b7a-a347-9c12ee847f5d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:24.897891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eaff8-f563-4b7a-a347-9c12ee847f5d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:24.898023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eaff8-f563-4b7a-a347-9c12ee847f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:24.968609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: INFO dbcounter [None req-940eaff8-f563-4b7a-a347-9c12ee847f5d None None] Registered counter for database keystone Aug 01 14:59:24.988759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG oslo_db.sqlalchemy.engines [None req-940eaff8-f563-4b7a-a347-9c12ee847f5d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80358) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 14:59:24.995121 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writer thread running {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 14:59:25.181544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG passlib.handlers.bcrypt [None req-940eaff8-f563-4b7a-a347-9c12ee847f5d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80358) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 14:59:25.181901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG passlib.handlers.bcrypt [None req-940eaff8-f563-4b7a-a347-9c12ee847f5d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80358) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 14:59:25.237031 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-940eaff8-f563-4b7a-a347-9c12ee847f5d None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:25.238695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-940eaff8-f563-4b7a-a347-9c12ee847f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:25.374095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 2/3] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:24 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 14:59:25.519321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5143a4-5eae-4294-a994-1d160cacc615 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:25.589341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: INFO dbcounter [None req-5f5143a4-5eae-4294-a994-1d160cacc615 None None] Registered counter for database keystone Aug 01 14:59:25.610687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG oslo_db.sqlalchemy.engines [None req-5f5143a4-5eae-4294-a994-1d160cacc615 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80359) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 14:59:25.616938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writer thread running {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 14:59:25.738302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5143a4-5eae-4294-a994-1d160cacc615 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:25.742840 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69fa1c-e39b-4120-8b63-fba6cd36e0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:25.744020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69fa1c-e39b-4120-8b63-fba6cd36e0ba admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:25.744287 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69fa1c-e39b-4120-8b63-fba6cd36e0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:25.744566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69fa1c-e39b-4120-8b63-fba6cd36e0ba admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:25.753191 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69fa1c-e39b-4120-8b63-fba6cd36e0ba admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:25.806964 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ea69fa1c-e39b-4120-8b63-fba6cd36e0ba admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 14:59:25.807245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69fa1c-e39b-4120-8b63-fba6cd36e0ba admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:25.817758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 2/4] 10.222.0.100 () {58 vars in 1329 bytes} [Mon Aug 1 14:59:25 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 14:59:25.823959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940eaff8-f563-4b7a-a347-9c12ee847f5d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:25.839541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940eaff8-f563-4b7a-a347-9c12ee847f5d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:25.844077 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac812a2-0bbe-4f34-959e-018e02baa035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:25.845224 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac812a2-0bbe-4f34-959e-018e02baa035 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:25.845533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac812a2-0bbe-4f34-959e-018e02baa035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:25.845824 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac812a2-0bbe-4f34-959e-018e02baa035 admin admin] PATH_INFO: `/v3/projects/bcc1ff25500841a3a7180e5fff03ff9f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:25.847429 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac812a2-0bbe-4f34-959e-018e02baa035 admin admin] RBAC: Authorizing `identity:get_project(project_id=bcc1ff25500841a3a7180e5fff03ff9f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:25.899038 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2ac812a2-0bbe-4f34-959e-018e02baa035 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 14:59:25.900216 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac812a2-0bbe-4f34-959e-018e02baa035 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:25.904757 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 3/5] 10.222.0.100 () {56 vars in 1212 bytes} [Mon Aug 1 14:59:25 2022] GET /identity/v3/projects/bcc1ff25500841a3a7180e5fff03ff9f => generated 345 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:27.908106 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef92646-5727-4170-a3a9-3587770b63c3 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:27.908659 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef92646-5727-4170-a3a9-3587770b63c3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:27.908659 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef92646-5727-4170-a3a9-3587770b63c3 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:27.912807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 3/6] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:27 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:27.918434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1180b65-b4cf-48f9-a97a-33904fee2a89 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:27.918921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1180b65-b4cf-48f9-a97a-33904fee2a89 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:27.919009 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1180b65-b4cf-48f9-a97a-33904fee2a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:27.938872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c1180b65-b4cf-48f9-a97a-33904fee2a89 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:27.940410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c1180b65-b4cf-48f9-a97a-33904fee2a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:27.948888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 4/7] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:27 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 14:59:27.971232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef92646-5727-4170-a3a9-3587770b63c3 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:27.984524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef92646-5727-4170-a3a9-3587770b63c3 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:27.988818 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c4090-3407-4349-93f9-0e854b1e94fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:27.989436 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c4090-3407-4349-93f9-0e854b1e94fd admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:27.989712 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c4090-3407-4349-93f9-0e854b1e94fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:27.989986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c4090-3407-4349-93f9-0e854b1e94fd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:27.998462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c4090-3407-4349-93f9-0e854b1e94fd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:28.000273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c4090-3407-4349-93f9-0e854b1e94fd admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:28.010133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 4/8] 10.222.0.100 () {58 vars in 1329 bytes} [Mon Aug 1 14:59:27 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 14:59:28.016071 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1180b65-b4cf-48f9-a97a-33904fee2a89 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:28.025206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1180b65-b4cf-48f9-a97a-33904fee2a89 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:28.029678 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ab7fd-4624-4557-bc28-961943689776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:28.030319 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ab7fd-4624-4557-bc28-961943689776 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:28.030710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ab7fd-4624-4557-bc28-961943689776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:28.030990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ab7fd-4624-4557-bc28-961943689776 admin admin] PATH_INFO: `/v3/users/24f06d52227a4059a53df8c2c849eefc` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:28.032376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ab7fd-4624-4557-bc28-961943689776 admin admin] RBAC: Authorizing `identity:get_user(user_id=24f06d52227a4059a53df8c2c849eefc)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:28.034211 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ab7fd-4624-4557-bc28-961943689776 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:28.035813 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 5/9] 10.222.0.100 () {56 vars in 1203 bytes} [Mon Aug 1 14:59:28 2022] GET /identity/v3/users/24f06d52227a4059a53df8c2c849eefc => generated 248 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:30.140708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b74c3c-48a8-498b-b875-941427ddd3c7 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.141247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b74c3c-48a8-498b-b875-941427ddd3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.141247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b74c3c-48a8-498b-b875-941427ddd3c7 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.142617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 5/10] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:30.148189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1f970-8e99-41b8-891f-2ae9a5eb88cf None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.148660 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1f970-8e99-41b8-891f-2ae9a5eb88cf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.148753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1f970-8e99-41b8-891f-2ae9a5eb88cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.160035 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aea646-1145-4082-9916-0bf7b85c93d1 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.160312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aea646-1145-4082-9916-0bf7b85c93d1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.160588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aea646-1145-4082-9916-0bf7b85c93d1 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.161927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 6/11] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 14:59:30.167326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfaf4c-00b6-4e8d-8f07-1a1c704634c8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.167615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfaf4c-00b6-4e8d-8f07-1a1c704634c8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.167894 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dfaf4c-00b6-4e8d-8f07-1a1c704634c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.172867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6fe1f970-8e99-41b8-891f-2ae9a5eb88cf None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:30.174425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6fe1f970-8e99-41b8-891f-2ae9a5eb88cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:30.213463 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG passlib.handlers.bcrypt [None req-b0dfaf4c-00b6-4e8d-8f07-1a1c704634c8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80359) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 14:59:30.213920 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG passlib.handlers.bcrypt [None req-b0dfaf4c-00b6-4e8d-8f07-1a1c704634c8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80359) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 14:59:30.260055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b0dfaf4c-00b6-4e8d-8f07-1a1c704634c8 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:30.261677 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b0dfaf4c-00b6-4e8d-8f07-1a1c704634c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:30.265166 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 6/12] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:59:30.288186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1f970-8e99-41b8-891f-2ae9a5eb88cf None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:30.314487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:30.376497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 7/13] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:59:30.398889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfaf4c-00b6-4e8d-8f07-1a1c704634c8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:30.420601 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1f970-8e99-41b8-891f-2ae9a5eb88cf None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:30.456784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38afb0-77d3-4c39-a692-aec31057a7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:30.457447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38afb0-77d3-4c39-a692-aec31057a7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.457728 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38afb0-77d3-4c39-a692-aec31057a7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.458012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38afb0-77d3-4c39-a692-aec31057a7d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.470001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38afb0-77d3-4c39-a692-aec31057a7d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-fc38afb0-77d3-4c39-a692-aec31057a7d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:30.479021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:30.480142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:30.481179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:30.481179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:30.481179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:30.481179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:30.481179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:30.481179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:30.481179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:30.481179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:30.481179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:30.481179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:30.481179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:30.481179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 14:59:30.481179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 7/14] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 14:59:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:30.488204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38afb0-77d3-4c39-a692-aec31057a7d9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:30.521924 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dfaf4c-00b6-4e8d-8f07-1a1c704634c8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:30.524521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:30.564152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bccc5-878c-4a85-8b78-78c562742772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:30.565040 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bccc5-878c-4a85-8b78-78c562742772 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.565332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bccc5-878c-4a85-8b78-78c562742772 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.565611 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bccc5-878c-4a85-8b78-78c562742772 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.580742 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:30.603335 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38afb0-77d3-4c39-a692-aec31057a7d9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:30.638208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c2fd4-2046-4df6-a977-71ad073aa0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:30.638870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c2fd4-2046-4df6-a977-71ad073aa0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.639160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c2fd4-2046-4df6-a977-71ad073aa0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.639438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c2fd4-2046-4df6-a977-71ad073aa0e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.640184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c2fd4-2046-4df6-a977-71ad073aa0e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:30.640489 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c2fd4-2046-4df6-a977-71ad073aa0e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:30.641898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c2fd4-2046-4df6-a977-71ad073aa0e3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:30.643231 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3bccc5-878c-4a85-8b78-78c562742772 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:30.645937 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3bccc5-878c-4a85-8b78-78c562742772 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:30.654719 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 8/15] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 14:59:30 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:30.660478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c2fd4-2046-4df6-a977-71ad073aa0e3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:30.753352 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 8/16] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 14:59:30 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:59:30.759034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bccc5-878c-4a85-8b78-78c562742772 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:30.762384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:30.779663 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c2fd4-2046-4df6-a977-71ad073aa0e3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:30.887582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bccc5-878c-4a85-8b78-78c562742772 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:30.924503 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a6a7e-a916-41a5-98bc-ea320a1b1161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:30.925082 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a6a7e-a916-41a5-98bc-ea320a1b1161 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.925365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a6a7e-a916-41a5-98bc-ea320a1b1161 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.925644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a6a7e-a916-41a5-98bc-ea320a1b1161 None admin] PATH_INFO: `/v3/users/24f06d52227a4059a53df8c2c849eefc` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.926873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a6a7e-a916-41a5-98bc-ea320a1b1161 None admin] RBAC: Authorizing `identity:get_user(user_id=24f06d52227a4059a53df8c2c849eefc)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:30.928361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a6a7e-a916-41a5-98bc-ea320a1b1161 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:30.929987 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 9/17] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:30 2022] GET /identity/v3/users/24f06d52227a4059a53df8c2c849eefc => generated 248 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:30.931473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99fb78a-df3b-48e5-808b-2c176ffecabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:30.932346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fb78a-df3b-48e5-808b-2c176ffecabb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:30.932630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fb78a-df3b-48e5-808b-2c176ffecabb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:30.932902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99fb78a-df3b-48e5-808b-2c176ffecabb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:30.936035 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a6a7e-a916-41a5-98bc-ea320a1b1161 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:30.956716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99fb78a-df3b-48e5-808b-2c176ffecabb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:30.958082 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:30.958266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99fb78a-df3b-48e5-808b-2c176ffecabb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-b99fb78a-df3b-48e5-808b-2c176ffecabb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:30.978684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:30.979600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:30.980784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:30.980784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:30.980784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:30.980784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 14:59:30.980784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 9/18] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 14:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:30.985840 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99fb78a-df3b-48e5-808b-2c176ffecabb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:31.047481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:31.060133 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a6a7e-a916-41a5-98bc-ea320a1b1161 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:31.093775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99fb78a-df3b-48e5-808b-2c176ffecabb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:31.096105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e33f65-7537-4e3a-b43e-9031df039351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:31.096904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e33f65-7537-4e3a-b43e-9031df039351 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:31.097299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e33f65-7537-4e3a-b43e-9031df039351 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:31.097667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e33f65-7537-4e3a-b43e-9031df039351 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:31.099437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e33f65-7537-4e3a-b43e-9031df039351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:31.102449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e33f65-7537-4e3a-b43e-9031df039351 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:31.109641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 10/19] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 14:59:30 2022] GET /identity/v3/domains/default => generated 200 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:31.110627 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e33f65-7537-4e3a-b43e-9031df039351 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:31.131902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae0eef-5e7f-450e-8a8c-96f8c3a09cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:31.132537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae0eef-5e7f-450e-8a8c-96f8c3a09cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:31.132820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae0eef-5e7f-450e-8a8c-96f8c3a09cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:31.134579 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae0eef-5e7f-450e-8a8c-96f8c3a09cb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:31.135348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae0eef-5e7f-450e-8a8c-96f8c3a09cb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:31.135657 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae0eef-5e7f-450e-8a8c-96f8c3a09cb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:31.137133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae0eef-5e7f-450e-8a8c-96f8c3a09cb2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:31.156271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 10/20] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 14:59:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:31.218321 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:31.239736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e33f65-7537-4e3a-b43e-9031df039351 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:31.278357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2dc823-2714-4a4f-9141-c885ee727b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:31.279039 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dc823-2714-4a4f-9141-c885ee727b21 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:31.279326 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dc823-2714-4a4f-9141-c885ee727b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:31.279604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dc823-2714-4a4f-9141-c885ee727b21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:31.280431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2dc823-2714-4a4f-9141-c885ee727b21 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.domain.id=default, user.id=24f06d52227a4059a53df8c2c849eefc) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:31.280748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2dc823-2714-4a4f-9141-c885ee727b21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:31.282240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2dc823-2714-4a4f-9141-c885ee727b21 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:31.290319 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 11/21] 10.222.0.100 () {56 vars in 1327 bytes} [Mon Aug 1 14:59:31 2022] GET /identity/v3/role_assignments?user.id=24f06d52227a4059a53df8c2c849eefc&scope.domain.id=default&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 234 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:33.937360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19360c7-92d9-4279-afd9-8b848a629d2f None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:33.937903 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19360c7-92d9-4279-afd9-8b848a629d2f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:33.937903 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19360c7-92d9-4279-afd9-8b848a629d2f None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:33.939486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 11/22] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:33.949950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae07de-bb0b-4e24-a9ae-aca227ff271e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:33.949950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae07de-bb0b-4e24-a9ae-aca227ff271e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:33.949950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae07de-bb0b-4e24-a9ae-aca227ff271e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:33.963694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d1b7a-d5ca-4073-966f-b9a8dbe67a76 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:33.963993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d1b7a-d5ca-4073-966f-b9a8dbe67a76 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:33.964254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d1b7a-d5ca-4073-966f-b9a8dbe67a76 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:33.966015 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 12/23] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:33 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:33.976158 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358f9d8-8b22-4a52-a0c5-28090ba27618 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:33.976426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358f9d8-8b22-4a52-a0c5-28090ba27618 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:33.976705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358f9d8-8b22-4a52-a0c5-28090ba27618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:33.984592 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e9ae07de-bb0b-4e24-a9ae-aca227ff271e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:33.992793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e9ae07de-bb0b-4e24-a9ae-aca227ff271e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:34.014384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-8358f9d8-8b22-4a52-a0c5-28090ba27618 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:34.015705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8358f9d8-8b22-4a52-a0c5-28090ba27618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:34.172610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 12/24] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:34.205271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 13/25] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:59:34.211477 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae07de-bb0b-4e24-a9ae-aca227ff271e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:34.257345 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358f9d8-8b22-4a52-a0c5-28090ba27618 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:34.358489 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:34.367043 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:34.411097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae07de-bb0b-4e24-a9ae-aca227ff271e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:34.470521 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9532ff-1554-4fcc-81fb-8a7055fc1c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:34.471124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9532ff-1554-4fcc-81fb-8a7055fc1c35 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:34.471399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9532ff-1554-4fcc-81fb-8a7055fc1c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:34.471681 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9532ff-1554-4fcc-81fb-8a7055fc1c35 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:34.472435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9532ff-1554-4fcc-81fb-8a7055fc1c35 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:34.473846 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9532ff-1554-4fcc-81fb-8a7055fc1c35 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:34.484830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358f9d8-8b22-4a52-a0c5-28090ba27618 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:34.592512 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394883f0-da89-4b9d-95f4-232e3a35a558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:34.593183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394883f0-da89-4b9d-95f4-232e3a35a558 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:34.593468 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394883f0-da89-4b9d-95f4-232e3a35a558 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:34.593745 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394883f0-da89-4b9d-95f4-232e3a35a558 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:34.615125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394883f0-da89-4b9d-95f4-232e3a35a558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:34.616637 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394883f0-da89-4b9d-95f4-232e3a35a558 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-394883f0-da89-4b9d-95f4-232e3a35a558 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 14:59:34.645428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:34.646324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:34.647601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 14:59:34.647601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 14:59:34.647601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 14:59:34.647601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 14:59:34.647601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 14/26] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 14:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:34.648222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c62e8-6eff-4f88-a694-e997320a1632 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:34.648222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c62e8-6eff-4f88-a694-e997320a1632 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:34.648508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c62e8-6eff-4f88-a694-e997320a1632 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:34.650272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 15/27] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:34.657860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862ca30-7b4b-40f9-87f4-ad18d8266a63 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:34.658236 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862ca30-7b4b-40f9-87f4-ad18d8266a63 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:34.658642 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862ca30-7b4b-40f9-87f4-ad18d8266a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:34.694112 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5862ca30-7b4b-40f9-87f4-ad18d8266a63 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:34.696834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5862ca30-7b4b-40f9-87f4-ad18d8266a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:34.714463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 13/28] 10.222.0.100 () {60 vars in 1136 bytes} [Mon Aug 1 14:59:34 2022] POST /identity/v3/roles => generated 220 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:34.715618 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9532ff-1554-4fcc-81fb-8a7055fc1c35 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:34.824145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 16/29] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:34.825926 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df03d75-32e1-4e67-ad10-e4b7b89c16a1 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:34.826418 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df03d75-32e1-4e67-ad10-e4b7b89c16a1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:34.826925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df03d75-32e1-4e67-ad10-e4b7b89c16a1 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:34.828564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 17/30] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:34 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:34.836506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cc7a3-6caf-41b0-9f3e-5d614858b03d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:34.837144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cc7a3-6caf-41b0-9f3e-5d614858b03d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:34.837619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88cc7a3-6caf-41b0-9f3e-5d614858b03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:34.845474 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9532ff-1554-4fcc-81fb-8a7055fc1c35 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:34.888117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ed281-0e0b-400a-90ce-e44b48e79822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:34.888790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ed281-0e0b-400a-90ce-e44b48e79822 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:34.889081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ed281-0e0b-400a-90ce-e44b48e79822 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:34.889362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ed281-0e0b-400a-90ce-e44b48e79822 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:34.890104 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ed281-0e0b-400a-90ce-e44b48e79822 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:34.890418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ed281-0e0b-400a-90ce-e44b48e79822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:34.892078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ed281-0e0b-400a-90ce-e44b48e79822 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:34.894159 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:34.934089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 14/31] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 14:59:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:34.939143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ed281-0e0b-400a-90ce-e44b48e79822 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:35.018996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f88cc7a3-6caf-41b0-9f3e-5d614858b03d None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:35.021155 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f88cc7a3-6caf-41b0-9f3e-5d614858b03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:35.050624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:35.100554 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ed281-0e0b-400a-90ce-e44b48e79822 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:35.117159 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 18/32] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:59:35.118983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cc7a3-6caf-41b0-9f3e-5d614858b03d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:35.158742 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c04517-3103-4cfb-86a8-57b34413583a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:35.159759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c04517-3103-4cfb-86a8-57b34413583a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.160243 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c04517-3103-4cfb-86a8-57b34413583a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.160706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c04517-3103-4cfb-86a8-57b34413583a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.174177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c04517-3103-4cfb-86a8-57b34413583a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-23c04517-3103-4cfb-86a8-57b34413583a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:35.179570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:35.180410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:35.182491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:35.182491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:35.182491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:35.182491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:35.182491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:35.182491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:35.182491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:35.182491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:35.182491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:35.182491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:35.182491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:35.182491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 14:59:35.182491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 15/33] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 14:59:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:35.183693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62b109-e09c-4b59-8be0-f7701570454e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.183980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62b109-e09c-4b59-8be0-f7701570454e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.184257 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d62b109-e09c-4b59-8be0-f7701570454e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.185787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 16/34] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:35.186262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62b109-e09c-4b59-8be0-f7701570454e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:35.259820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:35.262090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88cc7a3-6caf-41b0-9f3e-5d614858b03d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:35.311362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46daca09-6abd-414c-bb6a-1ba217c7a9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:35.311987 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46daca09-6abd-414c-bb6a-1ba217c7a9d1 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.312273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46daca09-6abd-414c-bb6a-1ba217c7a9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.312379 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d62b109-e09c-4b59-8be0-f7701570454e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:35.312576 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46daca09-6abd-414c-bb6a-1ba217c7a9d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.313529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46daca09-6abd-414c-bb6a-1ba217c7a9d1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:35.315547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46daca09-6abd-414c-bb6a-1ba217c7a9d1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:35.351946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40744a0c-2961-404f-8039-5c1c4ffb7902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:35.353026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40744a0c-2961-404f-8039-5c1c4ffb7902 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.353314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40744a0c-2961-404f-8039-5c1c4ffb7902 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.353591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40744a0c-2961-404f-8039-5c1c4ffb7902 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.354332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40744a0c-2961-404f-8039-5c1c4ffb7902 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:35.355787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40744a0c-2961-404f-8039-5c1c4ffb7902 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:35.374330 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 17/35] 10.222.0.100 () {60 vars in 1136 bytes} [Mon Aug 1 14:59:35 2022] POST /identity/v3/roles => generated 226 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:35.374922 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40744a0c-2961-404f-8039-5c1c4ffb7902 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:35.470709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 19/36] 10.222.0.100 () {60 vars in 1146 bytes} [Mon Aug 1 14:59:35 2022] POST /identity/v3/projects => generated 302 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:35.470709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf57cfb-d0f4-44b1-9334-c5205d11d1f6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.470709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf57cfb-d0f4-44b1-9334-c5205d11d1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.470709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf57cfb-d0f4-44b1-9334-c5205d11d1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.495746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-acf57cfb-d0f4-44b1-9334-c5205d11d1f6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:35.498316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-acf57cfb-d0f4-44b1-9334-c5205d11d1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:35.501635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40744a0c-2961-404f-8039-5c1c4ffb7902 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:35.503934 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:35.535109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:35.574231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf72ed-ce80-4b80-9e0b-ada2cf32d34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:35.575921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf72ed-ce80-4b80-9e0b-ada2cf32d34f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.575921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf72ed-ce80-4b80-9e0b-ada2cf32d34f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.575921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf72ed-ce80-4b80-9e0b-ada2cf32d34f None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.576262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf72ed-ce80-4b80-9e0b-ada2cf32d34f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:35.576577 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf72ed-ce80-4b80-9e0b-ada2cf32d34f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:35.578229 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcf72ed-ce80-4b80-9e0b-ada2cf32d34f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:35.590642 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 18/37] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 14:59:35 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:35.596615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2768519-86ab-45cd-9060-e22d1ba6f1c8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.597517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2768519-86ab-45cd-9060-e22d1ba6f1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.598974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2768519-86ab-45cd-9060-e22d1ba6f1c8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.604483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 19/38] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:35 2022] GET /identity => generated 271 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:35.609286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2768519-86ab-45cd-9060-e22d1ba6f1c8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:35.634691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 20/39] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:35.634691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b4b87-557f-4fa9-9795-386506b3aca2 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.634691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b4b87-557f-4fa9-9795-386506b3aca2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.634691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b4b87-557f-4fa9-9795-386506b3aca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.652865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ab5b4b87-557f-4fa9-9795-386506b3aca2 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:35.654346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ab5b4b87-557f-4fa9-9795-386506b3aca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:35.745289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 21/40] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:35.746245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b4b87-557f-4fa9-9795-386506b3aca2 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:35.763863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2768519-86ab-45cd-9060-e22d1ba6f1c8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:35.799354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3f34b-f1ae-4601-b392-d9bb53caa6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:35.800011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3f34b-f1ae-4601-b392-d9bb53caa6ed None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.800307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3f34b-f1ae-4601-b392-d9bb53caa6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.800591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3f34b-f1ae-4601-b392-d9bb53caa6ed None admin] PATH_INFO: `/v3/users/24f06d52227a4059a53df8c2c849eefc` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.801786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3f34b-f1ae-4601-b392-d9bb53caa6ed None admin] RBAC: Authorizing `identity:get_user(user_id=24f06d52227a4059a53df8c2c849eefc)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:35.803299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3f34b-f1ae-4601-b392-d9bb53caa6ed None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:35.804939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 20/41] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:35 2022] GET /identity/v3/users/24f06d52227a4059a53df8c2c849eefc => generated 248 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:35.805843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3f34b-f1ae-4601-b392-d9bb53caa6ed None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:35.828260 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:35.857509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:35.862531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b4b87-557f-4fa9-9795-386506b3aca2 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:35.901599 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f73f47-df88-4131-af04-e0d5b8661bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:35.902250 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f73f47-df88-4131-af04-e0d5b8661bf2 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.902535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f73f47-df88-4131-af04-e0d5b8661bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.902848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f73f47-df88-4131-af04-e0d5b8661bf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.903587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f73f47-df88-4131-af04-e0d5b8661bf2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:35.905121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f73f47-df88-4131-af04-e0d5b8661bf2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:35.915282 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3f34b-f1ae-4601-b392-d9bb53caa6ed None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:35.921032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 22/42] 10.222.0.100 () {60 vars in 1136 bytes} [Mon Aug 1 14:59:35 2022] POST /identity/v3/roles => generated 224 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:35.921885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f73f47-df88-4131-af04-e0d5b8661bf2 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:35.981848 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0227df-c1c3-4706-84ac-3a87ef981b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:35.982560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0227df-c1c3-4706-84ac-3a87ef981b3b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:35.983010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0227df-c1c3-4706-84ac-3a87ef981b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:35.983344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0227df-c1c3-4706-84ac-3a87ef981b3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:35.984836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0227df-c1c3-4706-84ac-3a87ef981b3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:35.986385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0227df-c1c3-4706-84ac-3a87ef981b3b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:35.988915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 21/43] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 14:59:35 2022] GET /identity/v3/domains/default => generated 200 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:35.997314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0227df-c1c3-4706-84ac-3a87ef981b3b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:36.058196 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f73f47-df88-4131-af04-e0d5b8661bf2 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:36.093571 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:36.104302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336e582-1611-4eca-9d38-7275fe62b62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:36.104302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336e582-1611-4eca-9d38-7275fe62b62d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:36.104842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336e582-1611-4eca-9d38-7275fe62b62d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:36.105007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336e582-1611-4eca-9d38-7275fe62b62d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:36.107038 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336e582-1611-4eca-9d38-7275fe62b62d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:36.109178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336e582-1611-4eca-9d38-7275fe62b62d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:36.110827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 23/44] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 14:59:35 2022] GET /identity/v3/domains/default => generated 200 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:36.117293 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336e582-1611-4eca-9d38-7275fe62b62d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:36.155642 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0227df-c1c3-4706-84ac-3a87ef981b3b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:36.172181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:36.201868 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ce231-fc31-4560-a6fc-813f6488c8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:36.202505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ce231-fc31-4560-a6fc-813f6488c8c2 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:36.202826 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ce231-fc31-4560-a6fc-813f6488c8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:36.203106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ce231-fc31-4560-a6fc-813f6488c8c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:36.203803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ce231-fc31-4560-a6fc-813f6488c8c2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:36.205361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ce231-fc31-4560-a6fc-813f6488c8c2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:36.235771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336e582-1611-4eca-9d38-7275fe62b62d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:36.236647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 22/45] 10.222.0.100 () {60 vars in 1146 bytes} [Mon Aug 1 14:59:35 2022] POST /identity/v3/projects => generated 313 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:36.276635 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1744f3-e1c4-4482-b039-daef890245fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:36.277334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1744f3-e1c4-4482-b039-daef890245fc None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:36.277690 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1744f3-e1c4-4482-b039-daef890245fc None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:36.277902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1744f3-e1c4-4482-b039-daef890245fc None admin] PATH_INFO: `/v3/domains/default/users/24f06d52227a4059a53df8c2c849eefc/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:36.280363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1744f3-e1c4-4482-b039-daef890245fc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=24f06d52227a4059a53df8c2c849eefc, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:36.281833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1744f3-e1c4-4482-b039-daef890245fc None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:36.300261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 24/46] 10.222.0.100 () {58 vars in 1366 bytes} [Mon Aug 1 14:59:36 2022] PUT /identity/v3/domains/default/users/24f06d52227a4059a53df8c2c849eefc/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:59:38.323487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bd0e5-c6af-4408-80d8-fb122ac158cf None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:38.324041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bd0e5-c6af-4408-80d8-fb122ac158cf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:38.324041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202bd0e5-c6af-4408-80d8-fb122ac158cf None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:38.325572 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 23/47] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:38.328682 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4ddbe-eda3-4c4a-b8ec-5b3060b4894b None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:38.329101 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4ddbe-eda3-4c4a-b8ec-5b3060b4894b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:38.329357 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4ddbe-eda3-4c4a-b8ec-5b3060b4894b None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:38.330350 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6155f-7950-419d-be0f-d98872a9d54b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:38.330646 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6155f-7950-419d-be0f-d98872a9d54b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:38.330769 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 25/48] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:38.330958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6155f-7950-419d-be0f-d98872a9d54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:38.335907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85c8c3-80dd-49ae-bbad-941c49bfdabb None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:38.336198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85c8c3-80dd-49ae-bbad-941c49bfdabb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:38.336483 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85c8c3-80dd-49ae-bbad-941c49bfdabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:38.350273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-58e6155f-7950-419d-be0f-d98872a9d54b None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:38.351864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-58e6155f-7950-419d-be0f-d98872a9d54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:38.355203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5c85c8c3-80dd-49ae-bbad-941c49bfdabb None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:38.356388 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5c85c8c3-80dd-49ae-bbad-941c49bfdabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:38.410786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:38.420106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 24/49] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:59:38.425822 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 26/50] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:59:38.442974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6155f-7950-419d-be0f-d98872a9d54b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:38.453832 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85c8c3-80dd-49ae-bbad-941c49bfdabb None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:38.538148 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:38.546632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6155f-7950-419d-be0f-d98872a9d54b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:38.562581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c85c8c3-80dd-49ae-bbad-941c49bfdabb None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:38.585200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37129748-3d17-4225-bf72-1313c510b480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:38.585771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37129748-3d17-4225-bf72-1313c510b480 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:38.586047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37129748-3d17-4225-bf72-1313c510b480 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:38.586332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37129748-3d17-4225-bf72-1313c510b480 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:38.597735 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e931c-4b6b-4806-89af-6ce5748591e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:38.598386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e931c-4b6b-4806-89af-6ce5748591e6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:38.598757 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e931c-4b6b-4806-89af-6ce5748591e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:38.599092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e931c-4b6b-4806-89af-6ce5748591e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:38.600263 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e931c-4b6b-4806-89af-6ce5748591e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:38.601726 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014e931c-4b6b-4806-89af-6ce5748591e6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:38.611065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 27/51] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 14:59:38 2022] GET /identity/v3/domains/default => generated 200 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:38.611224 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37129748-3d17-4225-bf72-1313c510b480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:38.611311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e931c-4b6b-4806-89af-6ce5748591e6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-37129748-3d17-4225-bf72-1313c510b480 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:38.612248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:38.613099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:38.613867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:38.613867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:38.613867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:38.613867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:38.613867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:38.613867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:38.613867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:38.613867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:38.613867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:38.613867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:38.613867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:38.613867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 14:59:38.614433 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 25/52] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 14:59:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:38.620800 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37129748-3d17-4225-bf72-1313c510b480 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:38.666592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:38.711833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e931c-4b6b-4806-89af-6ce5748591e6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:38.728442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37129748-3d17-4225-bf72-1313c510b480 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:38.754625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f24db-2176-49e7-9646-0054d4b56998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:38.755313 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f24db-2176-49e7-9646-0054d4b56998 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:38.755602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f24db-2176-49e7-9646-0054d4b56998 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:38.755880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3f24db-2176-49e7-9646-0054d4b56998 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:38.756600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f24db-2176-49e7-9646-0054d4b56998 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:38.758080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3f24db-2176-49e7-9646-0054d4b56998 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:38.768674 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f92b9d-5197-4cb7-b777-48956b6252e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:38.769622 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f92b9d-5197-4cb7-b777-48956b6252e7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:38.769967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f92b9d-5197-4cb7-b777-48956b6252e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:38.770303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f92b9d-5197-4cb7-b777-48956b6252e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:38.771305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f92b9d-5197-4cb7-b777-48956b6252e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:38.771747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f92b9d-5197-4cb7-b777-48956b6252e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:38.773426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f92b9d-5197-4cb7-b777-48956b6252e7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:38.782679 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 26/53] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 14:59:38 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:38.788671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f92b9d-5197-4cb7-b777-48956b6252e7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:38.790253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 28/54] 10.222.0.100 () {60 vars in 1145 bytes} [Mon Aug 1 14:59:38 2022] POST /identity/v3/projects => generated 299 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:59:38.809474 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:38.900715 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f92b9d-5197-4cb7-b777-48956b6252e7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:38.939820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d8b6a-7ae1-44e6-8857-9e328a889623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:38.940481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d8b6a-7ae1-44e6-8857-9e328a889623 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:38.940766 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d8b6a-7ae1-44e6-8857-9e328a889623 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:38.941044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d8b6a-7ae1-44e6-8857-9e328a889623 None admin] PATH_INFO: `/v3/users/24f06d52227a4059a53df8c2c849eefc` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:38.942261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d8b6a-7ae1-44e6-8857-9e328a889623 None admin] RBAC: Authorizing `identity:get_user(user_id=24f06d52227a4059a53df8c2c849eefc)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:38.943756 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d8b6a-7ae1-44e6-8857-9e328a889623 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:38.945382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 27/55] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:38 2022] GET /identity/v3/users/24f06d52227a4059a53df8c2c849eefc => generated 248 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:38.951206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f24db-2176-49e7-9646-0054d4b56998 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:39.070284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3f24db-2176-49e7-9646-0054d4b56998 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:39.081222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:39.117242 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f7399-e368-43c9-a333-39f12259a523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:39.117911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f7399-e368-43c9-a333-39f12259a523 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:39.118820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f7399-e368-43c9-a333-39f12259a523 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:39.119114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f7399-e368-43c9-a333-39f12259a523 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:39.120428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f7399-e368-43c9-a333-39f12259a523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:39.121915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f7399-e368-43c9-a333-39f12259a523 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:39.123920 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 29/56] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 14:59:38 2022] GET /identity/v3/domains/default => generated 200 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:39.131958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d8b6a-7ae1-44e6-8857-9e328a889623 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:39.224694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:39.242469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d8b6a-7ae1-44e6-8857-9e328a889623 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:39.277040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e0814-960c-4d53-83a8-b5ee5cafc5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:39.277663 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e0814-960c-4d53-83a8-b5ee5cafc5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:39.277947 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e0814-960c-4d53-83a8-b5ee5cafc5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:39.278224 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e0814-960c-4d53-83a8-b5ee5cafc5b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:39.279048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e0814-960c-4d53-83a8-b5ee5cafc5b2 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.domain.id=default, user.id=24f06d52227a4059a53df8c2c849eefc) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:39.279367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e0814-960c-4d53-83a8-b5ee5cafc5b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:39.280846 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e0814-960c-4d53-83a8-b5ee5cafc5b2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:39.288105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 28/57] 10.222.0.100 () {56 vars in 1327 bytes} [Mon Aug 1 14:59:39 2022] GET /identity/v3/role_assignments?user.id=24f06d52227a4059a53df8c2c849eefc&scope.domain.id=default&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 532 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:40.846526 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117a09a-85a2-4532-afb1-452486a81e69 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:40.847069 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117a09a-85a2-4532-afb1-452486a81e69 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:40.847177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117a09a-85a2-4532-afb1-452486a81e69 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:40.848498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 30/58] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:40.853543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f11d23-6b5f-4d19-9da9-51d5f70344a8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:40.853956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f11d23-6b5f-4d19-9da9-51d5f70344a8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:40.854101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f11d23-6b5f-4d19-9da9-51d5f70344a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:40.872305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-53f11d23-6b5f-4d19-9da9-51d5f70344a8 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:40.873491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-53f11d23-6b5f-4d19-9da9-51d5f70344a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:40.937253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 29/59] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:40.962839 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117a09a-85a2-4532-afb1-452486a81e69 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:41.066784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117a09a-85a2-4532-afb1-452486a81e69 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:41.100146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1c31d-00a6-4881-bf4d-9bf9bfed96f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:41.100708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1c31d-00a6-4881-bf4d-9bf9bfed96f7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:41.100992 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1c31d-00a6-4881-bf4d-9bf9bfed96f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:41.101268 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1c31d-00a6-4881-bf4d-9bf9bfed96f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:41.102974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1c31d-00a6-4881-bf4d-9bf9bfed96f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:41.104463 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1c31d-00a6-4881-bf4d-9bf9bfed96f7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:41.106049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 31/60] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 14:59:40 2022] GET /identity/v3/domains/default => generated 200 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:41.111934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f11d23-6b5f-4d19-9da9-51d5f70344a8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:41.197002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:41.218931 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f11d23-6b5f-4d19-9da9-51d5f70344a8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:41.253291 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171786bb-78da-4e29-bf72-823421909b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:41.253918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171786bb-78da-4e29-bf72-823421909b5d None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:41.254205 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171786bb-78da-4e29-bf72-823421909b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:41.254481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171786bb-78da-4e29-bf72-823421909b5d None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:41.255209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171786bb-78da-4e29-bf72-823421909b5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:41.256667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171786bb-78da-4e29-bf72-823421909b5d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:41.283042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 30/61] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 14:59:41 2022] POST /identity/v3/users => generated 276 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:44.569092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6358183-6704-4679-a5b5-583fc1884c5c None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:44.570106 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6358183-6704-4679-a5b5-583fc1884c5c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:44.570595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6358183-6704-4679-a5b5-583fc1884c5c None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:44.572796 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 32/62] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:44 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:44.578769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23795f0-66bb-457e-a4c1-94368f1e6ca7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:44.579727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23795f0-66bb-457e-a4c1-94368f1e6ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:44.580223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23795f0-66bb-457e-a4c1-94368f1e6ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:44.607026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f23795f0-66bb-457e-a4c1-94368f1e6ca7 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:44.608607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f23795f0-66bb-457e-a4c1-94368f1e6ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:44.707968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 31/63] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:44.715776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791e62e-1efe-40ac-87c6-ee9cd83ab136 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:44.716471 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791e62e-1efe-40ac-87c6-ee9cd83ab136 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:44.716955 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791e62e-1efe-40ac-87c6-ee9cd83ab136 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:44.718555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 33/64] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:44 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:44.722318 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41ce7f-3ece-4cca-a6ec-aa9d9ea3b5df None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:44.722887 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41ce7f-3ece-4cca-a6ec-aa9d9ea3b5df None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:44.723394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41ce7f-3ece-4cca-a6ec-aa9d9ea3b5df None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:44.725171 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd90dca-58d0-477e-8894-d1e1131bd041 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:44.725660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd90dca-58d0-477e-8894-d1e1131bd041 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:44.726158 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd90dca-58d0-477e-8894-d1e1131bd041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:44.733939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 32/65] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:44 2022] GET /identity => generated 271 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:44.737758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41ce7f-3ece-4cca-a6ec-aa9d9ea3b5df None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:44.749158 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-8dd90dca-58d0-477e-8894-d1e1131bd041 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:44.750409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8dd90dca-58d0-477e-8894-d1e1131bd041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:44.811565 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:44.849606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 34/66] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:59:44.850956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8124-40f4-4e34-b048-5d8156b20892 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:44.851260 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8124-40f4-4e34-b048-5d8156b20892 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:44.851549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8124-40f4-4e34-b048-5d8156b20892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:44.855414 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:44.872026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41ce7f-3ece-4cca-a6ec-aa9d9ea3b5df None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:44.873495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-eb8f8124-40f4-4e34-b048-5d8156b20892 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:44.875139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-eb8f8124-40f4-4e34-b048-5d8156b20892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:44.916085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623be3b-9d41-4a00-8d1f-b9c3307ba4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:44.916982 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623be3b-9d41-4a00-8d1f-b9c3307ba4fc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:44.917398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623be3b-9d41-4a00-8d1f-b9c3307ba4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:44.917879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623be3b-9d41-4a00-8d1f-b9c3307ba4fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:44.930364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623be3b-9d41-4a00-8d1f-b9c3307ba4fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e623be3b-9d41-4a00-8d1f-b9c3307ba4fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:44.936934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:44.937905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:44.938727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:44.938727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:44.938727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:44.938727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:44.938727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:44.938727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:44.938727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:44.938727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:44.938727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:44.938727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:44.938727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:59:44.938727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 14:59:44.941288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 33/67] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 14:59:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:44.942430 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623be3b-9d41-4a00-8d1f-b9c3307ba4fc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:44.978384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 35/68] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:44.980549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f8124-40f4-4e34-b048-5d8156b20892 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:45.084558 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623be3b-9d41-4a00-8d1f-b9c3307ba4fc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:45.112142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:45.116702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f8124-40f4-4e34-b048-5d8156b20892 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:45.119902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:45.126852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0457e2-ed0c-4f59-aeb2-28a69e87b2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:45.127740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0457e2-ed0c-4f59-aeb2-28a69e87b2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:45.128184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0457e2-ed0c-4f59-aeb2-28a69e87b2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:45.128472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0457e2-ed0c-4f59-aeb2-28a69e87b2c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:45.139722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0457e2-ed0c-4f59-aeb2-28a69e87b2c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1a0457e2-ed0c-4f59-aeb2-28a69e87b2c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:45.144991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:45.146545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:45.147989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:45.147989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:45.147989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:45.147989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:45.147989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:45.147989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:45.147989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:45.147989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:45.147989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:45.147989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:45.147989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:45.147989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 14:59:45.147989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 34/69] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 14:59:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:45.147989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0457e2-ed0c-4f59-aeb2-28a69e87b2c2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:45.161088 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63b03e-843d-49cc-b62c-8b7d5f1c564c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:45.161723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63b03e-843d-49cc-b62c-8b7d5f1c564c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:45.162006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63b03e-843d-49cc-b62c-8b7d5f1c564c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:45.162284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63b03e-843d-49cc-b62c-8b7d5f1c564c None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:45.163078 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63b03e-843d-49cc-b62c-8b7d5f1c564c None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:45.163388 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63b03e-843d-49cc-b62c-8b7d5f1c564c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:45.165332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63b03e-843d-49cc-b62c-8b7d5f1c564c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:45.177784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 36/70] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 14:59:44 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:45.178886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396be41-fa45-46f0-bdaa-65795f471e1c None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:45.179181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396be41-fa45-46f0-bdaa-65795f471e1c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:45.179464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396be41-fa45-46f0-bdaa-65795f471e1c None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:45.181265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 37/71] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:45.186124 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396be41-fa45-46f0-bdaa-65795f471e1c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:45.261045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0457e2-ed0c-4f59-aeb2-28a69e87b2c2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:45.291940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396be41-fa45-46f0-bdaa-65795f471e1c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:45.296337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f0d8c-e784-4026-bf60-b636539e886b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:45.296982 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f0d8c-e784-4026-bf60-b636539e886b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:45.297267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f0d8c-e784-4026-bf60-b636539e886b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:45.297545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f0d8c-e784-4026-bf60-b636539e886b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:45.307837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f0d8c-e784-4026-bf60-b636539e886b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-426f0d8c-e784-4026-bf60-b636539e886b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:45.312758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:45.313649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:45.314455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:45.314455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:45.314455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:45.314455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:45.314455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:45.314455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:45.314455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:45.314455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:45.314455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:45.314455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:45.314455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:59:45.314455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 14:59:45.314455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 35/72] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 14:59:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:45.315522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f0d8c-e784-4026-bf60-b636539e886b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:45.328441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b424d-df4e-4630-873e-b58454535933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:45.329064 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b424d-df4e-4630-873e-b58454535933 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:45.329346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b424d-df4e-4630-873e-b58454535933 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:45.329621 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b424d-df4e-4630-873e-b58454535933 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:45.330339 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b424d-df4e-4630-873e-b58454535933 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:45.330642 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b424d-df4e-4630-873e-b58454535933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:45.332053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b424d-df4e-4630-873e-b58454535933 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:45.341571 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 38/73] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 14:59:45 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:45.342723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fff31f-e2fd-44bf-8fc2-8e91498d0c3c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:45.343018 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fff31f-e2fd-44bf-8fc2-8e91498d0c3c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:45.343299 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fff31f-e2fd-44bf-8fc2-8e91498d0c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:45.361987 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b9fff31f-e2fd-44bf-8fc2-8e91498d0c3c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:45.363252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b9fff31f-e2fd-44bf-8fc2-8e91498d0c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:45.366433 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:45.388477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:45.429133 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f0d8c-e784-4026-bf60-b636539e886b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:45.434234 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 39/74] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:45.437159 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e01b7a-7d57-46e5-a850-111591d4b316 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:45.437459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e01b7a-7d57-46e5-a850-111591d4b316 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:45.437826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e01b7a-7d57-46e5-a850-111591d4b316 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:45.439758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 40/75] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:45 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:45.440465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e01b7a-7d57-46e5-a850-111591d4b316 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:45.469271 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef7675-638b-4ce2-9264-22870a8e6873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:45.469927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef7675-638b-4ce2-9264-22870a8e6873 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:45.470209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef7675-638b-4ce2-9264-22870a8e6873 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:45.470487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef7675-638b-4ce2-9264-22870a8e6873 None admin] PATH_INFO: `/v3/users/9d8a17c4135b4de4bf1d369d28f1c797` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:45.485050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef7675-638b-4ce2-9264-22870a8e6873 None admin] RBAC: Authorizing `identity:get_user(user_id=9d8a17c4135b4de4bf1d369d28f1c797)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:45.486626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef7675-638b-4ce2-9264-22870a8e6873 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:45.488454 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 36/76] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:45 2022] GET /identity/v3/users/9d8a17c4135b4de4bf1d369d28f1c797 => generated 276 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:45.489148 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef7675-638b-4ce2-9264-22870a8e6873 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:45.552056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e01b7a-7d57-46e5-a850-111591d4b316 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:45.590336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed7bf0-2638-4e34-9182-cafe94160dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:45.591023 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed7bf0-2638-4e34-9182-cafe94160dcc None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:45.591260 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed7bf0-2638-4e34-9182-cafe94160dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:45.591537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed7bf0-2638-4e34-9182-cafe94160dcc None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:45.592257 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed7bf0-2638-4e34-9182-cafe94160dcc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:45.592566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed7bf0-2638-4e34-9182-cafe94160dcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:45.593961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed7bf0-2638-4e34-9182-cafe94160dcc None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:45.599679 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef7675-638b-4ce2-9264-22870a8e6873 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:45.601217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:45.602180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 41/77] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 14:59:45 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:45.608371 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c0d66-2087-45b4-a4b0-f17d7f0af51c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:45.608672 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c0d66-2087-45b4-a4b0-f17d7f0af51c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:45.608954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c0d66-2087-45b4-a4b0-f17d7f0af51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:45.626775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ef2c0d66-2087-45b4-a4b0-f17d7f0af51c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:45.627995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ef2c0d66-2087-45b4-a4b0-f17d7f0af51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:45.636467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73888074-9ce2-4621-b07d-587bcb4e154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:45.637127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73888074-9ce2-4621-b07d-587bcb4e154c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:45.637415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73888074-9ce2-4621-b07d-587bcb4e154c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:45.637697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73888074-9ce2-4621-b07d-587bcb4e154c None admin] PATH_INFO: `/v3/users/24f06d52227a4059a53df8c2c849eefc` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:45.638915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73888074-9ce2-4621-b07d-587bcb4e154c None admin] RBAC: Authorizing `identity:get_user(user_id=24f06d52227a4059a53df8c2c849eefc)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:45.640370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73888074-9ce2-4621-b07d-587bcb4e154c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:45.642155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 37/78] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:45 2022] GET /identity/v3/users/24f06d52227a4059a53df8c2c849eefc => generated 248 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:45.642551 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:45.642869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73888074-9ce2-4621-b07d-587bcb4e154c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:45.705388 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 42/79] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:45.706254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c0d66-2087-45b4-a4b0-f17d7f0af51c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:45.761398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73888074-9ce2-4621-b07d-587bcb4e154c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:45.797632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cab4c7-a0f4-4d94-8202-3f8f376a1b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:45.798291 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cab4c7-a0f4-4d94-8202-3f8f376a1b07 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:45.798576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cab4c7-a0f4-4d94-8202-3f8f376a1b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:45.798894 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cab4c7-a0f4-4d94-8202-3f8f376a1b07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:45.809607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cab4c7-a0f4-4d94-8202-3f8f376a1b07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:45.814502 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2c0d66-2087-45b4-a4b0-f17d7f0af51c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-89cab4c7-a0f4-4d94-8202-3f8f376a1b07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:45.814624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:45.815669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:45.816465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:45.816465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:45.816465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:45.816465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:45.816465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:45.816465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:45.816465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:45.816465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:45.816465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:45.816465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:45.816465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:59:45.816465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 14:59:45.816465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 38/80] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 14:59:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:45.817109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cab4c7-a0f4-4d94-8202-3f8f376a1b07 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:45.849175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:45.856300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff471fb-f2a8-4a6b-ae7a-0f42dfc00881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:45.857034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff471fb-f2a8-4a6b-ae7a-0f42dfc00881 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:45.857322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff471fb-f2a8-4a6b-ae7a-0f42dfc00881 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:45.857594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff471fb-f2a8-4a6b-ae7a-0f42dfc00881 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:45.878084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:45.923325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cab4c7-a0f4-4d94-8202-3f8f376a1b07 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:45.929746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff471fb-f2a8-4a6b-ae7a-0f42dfc00881 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:45.931302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff471fb-f2a8-4a6b-ae7a-0f42dfc00881 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:45.958569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bc6c3-d16f-42b9-ad05-87c4a2a73c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:45.959252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bc6c3-d16f-42b9-ad05-87c4a2a73c20 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:45.959540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bc6c3-d16f-42b9-ad05-87c4a2a73c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:45.959817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56bc6c3-d16f-42b9-ad05-87c4a2a73c20 None admin] PATH_INFO: `/v3/users/9d8a17c4135b4de4bf1d369d28f1c797` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:45.961012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56bc6c3-d16f-42b9-ad05-87c4a2a73c20 None admin] RBAC: Authorizing `identity:get_user(user_id=9d8a17c4135b4de4bf1d369d28f1c797)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:45.963413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56bc6c3-d16f-42b9-ad05-87c4a2a73c20 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:45.965023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 39/81] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:45 2022] GET /identity/v3/users/9d8a17c4135b4de4bf1d369d28f1c797 => generated 276 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:45.969916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bc6c3-d16f-42b9-ad05-87c4a2a73c20 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:46.035116 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 43/82] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 14:59:45 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:59:46.035976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff471fb-f2a8-4a6b-ae7a-0f42dfc00881 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:46.076594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:46.081518 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56bc6c3-d16f-42b9-ad05-87c4a2a73c20 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:46.086010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:46.116151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a99442-6c1b-4b47-919e-761be4978538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:46.116694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a99442-6c1b-4b47-919e-761be4978538 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:46.116973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a99442-6c1b-4b47-919e-761be4978538 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:46.117265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a99442-6c1b-4b47-919e-761be4978538 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:46.139143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff471fb-f2a8-4a6b-ae7a-0f42dfc00881 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:46.173910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65faf1-468f-4c76-b11f-cda3728359d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:46.174560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65faf1-468f-4c76-b11f-cda3728359d9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:46.174881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65faf1-468f-4c76-b11f-cda3728359d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:46.175169 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65faf1-468f-4c76-b11f-cda3728359d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:46.176329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65faf1-468f-4c76-b11f-cda3728359d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:46.177787 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65faf1-468f-4c76-b11f-cda3728359d9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:46.179424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 44/83] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 14:59:46 2022] GET /identity/v3/domains/default => generated 200 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:46.180198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65faf1-468f-4c76-b11f-cda3728359d9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:46.192222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a99442-6c1b-4b47-919e-761be4978538 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:46.193734 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a99442-6c1b-4b47-919e-761be4978538 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:46.289280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:46.291011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65faf1-468f-4c76-b11f-cda3728359d9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:46.295493 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 40/84] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 14:59:45 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:59:46.297015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a99442-6c1b-4b47-919e-761be4978538 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:46.314307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:46.334517 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b180b8-4c4d-4e4b-b0d1-0e2810c89777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:46.335183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b180b8-4c4d-4e4b-b0d1-0e2810c89777 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:46.335469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b180b8-4c4d-4e4b-b0d1-0e2810c89777 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:46.335750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b180b8-4c4d-4e4b-b0d1-0e2810c89777 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:46.336478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b180b8-4c4d-4e4b-b0d1-0e2810c89777 None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:46.336787 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b180b8-4c4d-4e4b-b0d1-0e2810c89777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:46.338441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b180b8-4c4d-4e4b-b0d1-0e2810c89777 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:46.346968 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 45/85] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 14:59:46 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:46.347625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b180b8-4c4d-4e4b-b0d1-0e2810c89777 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:46.409235 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a99442-6c1b-4b47-919e-761be4978538 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:46.445839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7235f-591d-476e-ab69-43d904f19ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:46.446495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7235f-591d-476e-ab69-43d904f19ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:46.446817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7235f-591d-476e-ab69-43d904f19ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:46.447097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7235f-591d-476e-ab69-43d904f19ff2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:46.458207 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b180b8-4c4d-4e4b-b0d1-0e2810c89777 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:46.497733 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31845323-eb71-4b72-9986-7e765a04d0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:46.498385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31845323-eb71-4b72-9986-7e765a04d0af None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:46.498690 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31845323-eb71-4b72-9986-7e765a04d0af None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:46.498961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31845323-eb71-4b72-9986-7e765a04d0af None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:46.500160 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31845323-eb71-4b72-9986-7e765a04d0af None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:46.501615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31845323-eb71-4b72-9986-7e765a04d0af None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:46.503323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 46/86] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 14:59:46 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:46.504141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31845323-eb71-4b72-9986-7e765a04d0af None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:46.520997 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:46.525730 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7235f-591d-476e-ab69-43d904f19ff2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:46.527284 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7235f-591d-476e-ab69-43d904f19ff2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:46.558238 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:46.611225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31845323-eb71-4b72-9986-7e765a04d0af None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:46.640111 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 41/87] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 14:59:46 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:59:46.640970 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7235f-591d-476e-ab69-43d904f19ff2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:46.647809 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1dad7-5200-40b4-9a4f-c4acc7ec2252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:46.648443 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1dad7-5200-40b4-9a4f-c4acc7ec2252 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:46.648732 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1dad7-5200-40b4-9a4f-c4acc7ec2252 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:46.649117 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b1dad7-5200-40b4-9a4f-c4acc7ec2252 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:46.649819 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1dad7-5200-40b4-9a4f-c4acc7ec2252 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:46.651329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b1dad7-5200-40b4-9a4f-c4acc7ec2252 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:46.683893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 47/88] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 14:59:46 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:46.684838 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1dad7-5200-40b4-9a4f-c4acc7ec2252 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:46.759431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:46.763926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7235f-591d-476e-ab69-43d904f19ff2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:46.811658 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cd872-064a-4bc2-a871-dbb675cd2a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:46.812326 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cd872-064a-4bc2-a871-dbb675cd2a1c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:46.812608 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cd872-064a-4bc2-a871-dbb675cd2a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:46.812889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cd872-064a-4bc2-a871-dbb675cd2a1c None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:46.814194 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7cd872-064a-4bc2-a871-dbb675cd2a1c None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:46.815996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7cd872-064a-4bc2-a871-dbb675cd2a1c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:46.822921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 42/89] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 14:59:46 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:46.822921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cd872-064a-4bc2-a871-dbb675cd2a1c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:46.841611 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b1dad7-5200-40b4-9a4f-c4acc7ec2252 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:46.844342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:46.884117 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c5d58-fa5e-43e9-9268-efab1d7b85b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:46.886648 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5d58-fa5e-43e9-9268-efab1d7b85b3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:46.886648 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5d58-fa5e-43e9-9268-efab1d7b85b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:46.886648 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5d58-fa5e-43e9-9268-efab1d7b85b3 None admin] PATH_INFO: `/v3/users/9d8a17c4135b4de4bf1d369d28f1c797` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:46.886648 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c5d58-fa5e-43e9-9268-efab1d7b85b3 None admin] RBAC: Authorizing `identity:get_user(user_id=9d8a17c4135b4de4bf1d369d28f1c797)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:46.887983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c5d58-fa5e-43e9-9268-efab1d7b85b3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:46.889600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 48/90] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:46 2022] GET /identity/v3/users/9d8a17c4135b4de4bf1d369d28f1c797 => generated 276 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:46.890386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c5d58-fa5e-43e9-9268-efab1d7b85b3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:46.953365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cd872-064a-4bc2-a871-dbb675cd2a1c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:46.999769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee9533-6205-416b-8065-3500605c4480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:47.000402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee9533-6205-416b-8065-3500605c4480 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:47.000692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee9533-6205-416b-8065-3500605c4480 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:47.000975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee9533-6205-416b-8065-3500605c4480 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:47.001785 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee9533-6205-416b-8065-3500605c4480 None admin] RBAC: Adding query filter params (role.id=d1f6973ce11a4644b649b38f8b2df173, scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80, user.id=9d8a17c4135b4de4bf1d369d28f1c797) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:47.002101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee9533-6205-416b-8065-3500605c4480 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:47.004115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee9533-6205-416b-8065-3500605c4480 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:47.010568 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 43/91] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 14:59:46 2022] GET /identity/v3/role_assignments?user.id=9d8a17c4135b4de4bf1d369d28f1c797&scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80&role.id=d1f6973ce11a4644b649b38f8b2df173 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:47.011411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee9533-6205-416b-8065-3500605c4480 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:47.022913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c5d58-fa5e-43e9-9268-efab1d7b85b3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:47.050086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:47.065270 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac592310-6b31-4faf-b7a1-da856cc5e8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:47.065934 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac592310-6b31-4faf-b7a1-da856cc5e8ed None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:47.066219 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac592310-6b31-4faf-b7a1-da856cc5e8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:47.066501 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac592310-6b31-4faf-b7a1-da856cc5e8ed None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:47.067748 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac592310-6b31-4faf-b7a1-da856cc5e8ed None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:47.069293 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac592310-6b31-4faf-b7a1-da856cc5e8ed None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:47.071761 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 49/92] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 14:59:46 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:47.072574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac592310-6b31-4faf-b7a1-da856cc5e8ed None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:47.128673 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:47.158732 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee9533-6205-416b-8065-3500605c4480 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:47.204493 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126018e3-6d4b-4071-93fc-e17d8e092815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:47.205068 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126018e3-6d4b-4071-93fc-e17d8e092815 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:47.205357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126018e3-6d4b-4071-93fc-e17d8e092815 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:47.205638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126018e3-6d4b-4071-93fc-e17d8e092815 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:47.206520 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126018e3-6d4b-4071-93fc-e17d8e092815 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80, user.id=24f06d52227a4059a53df8c2c849eefc) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:47.206885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126018e3-6d4b-4071-93fc-e17d8e092815 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:47.208389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126018e3-6d4b-4071-93fc-e17d8e092815 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:47.216912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 44/93] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 14:59:47 2022] GET /identity/v3/role_assignments?user.id=24f06d52227a4059a53df8c2c849eefc&scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:47.217819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126018e3-6d4b-4071-93fc-e17d8e092815 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:47.295306 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac592310-6b31-4faf-b7a1-da856cc5e8ed None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:47.340870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8e573-9bcb-4601-b643-533a2497be92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:47.341503 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8e573-9bcb-4601-b643-533a2497be92 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:47.341788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8e573-9bcb-4601-b643-533a2497be92 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:47.342066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8e573-9bcb-4601-b643-533a2497be92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:47.344141 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:47.353895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126018e3-6d4b-4071-93fc-e17d8e092815 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:47.446573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8123e-7414-4bc2-a2c6-c3e03161827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:47.447244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8123e-7414-4bc2-a2c6-c3e03161827d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:47.447529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8123e-7414-4bc2-a2c6-c3e03161827d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:47.447808 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8123e-7414-4bc2-a2c6-c3e03161827d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:47.448679 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8123e-7414-4bc2-a2c6-c3e03161827d None admin] RBAC: Adding query filter params (role.id=57221d717a454c7fae4f2df1de9dbe5b, scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80, user.id=9d8a17c4135b4de4bf1d369d28f1c797) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:47.448999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8123e-7414-4bc2-a2c6-c3e03161827d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:47.450488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8123e-7414-4bc2-a2c6-c3e03161827d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:47.466815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 45/94] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 14:59:47 2022] GET /identity/v3/role_assignments?user.id=9d8a17c4135b4de4bf1d369d28f1c797&scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80&role.id=57221d717a454c7fae4f2df1de9dbe5b => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:47.489874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8e573-9bcb-4601-b643-533a2497be92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:47.491439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8e573-9bcb-4601-b643-533a2497be92 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:47.557910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:47.648566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 50/95] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 14:59:47 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:59:47.654547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8123e-7414-4bc2-a2c6-c3e03161827d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:47.791612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8123e-7414-4bc2-a2c6-c3e03161827d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:47.837630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3e57f-7eb8-4c62-8327-3f42228ddf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:47.838550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3e57f-7eb8-4c62-8327-3f42228ddf53 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:47.838905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3e57f-7eb8-4c62-8327-3f42228ddf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:47.839194 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3e57f-7eb8-4c62-8327-3f42228ddf53 None admin] PATH_INFO: `/v3/projects/0ab4610904d94edd8eb3a1f420c13314` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:47.840750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3e57f-7eb8-4c62-8327-3f42228ddf53 None admin] RBAC: Authorizing `identity:get_project(project_id=0ab4610904d94edd8eb3a1f420c13314)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:47.842213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3e57f-7eb8-4c62-8327-3f42228ddf53 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:47.843930 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 46/96] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 14:59:47 2022] GET /identity/v3/projects/0ab4610904d94edd8eb3a1f420c13314 => generated 313 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:47.849968 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8e573-9bcb-4601-b643-533a2497be92 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:47.968609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8e573-9bcb-4601-b643-533a2497be92 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:48.010587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e7a9f-aed0-4a00-ac42-0f63f7292262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:48.011249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e7a9f-aed0-4a00-ac42-0f63f7292262 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:48.011537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e7a9f-aed0-4a00-ac42-0f63f7292262 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:48.011821 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e7a9f-aed0-4a00-ac42-0f63f7292262 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:48.012620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e7a9f-aed0-4a00-ac42-0f63f7292262 None admin] RBAC: Adding query filter params (role.id=d1f6973ce11a4644b649b38f8b2df173, scope.project.id=0ab4610904d94edd8eb3a1f420c13314, user.id=9d8a17c4135b4de4bf1d369d28f1c797) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:48.012941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e7a9f-aed0-4a00-ac42-0f63f7292262 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:48.014406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e7a9f-aed0-4a00-ac42-0f63f7292262 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:48.023246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 51/97] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 14:59:47 2022] GET /identity/v3/role_assignments?user.id=9d8a17c4135b4de4bf1d369d28f1c797&scope.project.id=0ab4610904d94edd8eb3a1f420c13314&role.id=d1f6973ce11a4644b649b38f8b2df173 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:49.728265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160ad0b-c30b-4001-beac-e7872e872336 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:49.728771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160ad0b-c30b-4001-beac-e7872e872336 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:49.728843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5160ad0b-c30b-4001-beac-e7872e872336 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:49.730508 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 47/98] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:49 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:49.735785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14eafc-2977-46fe-8cf6-4be2a18b5b21 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:49.736195 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14eafc-2977-46fe-8cf6-4be2a18b5b21 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:49.736356 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14eafc-2977-46fe-8cf6-4be2a18b5b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:49.749584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:49.761201 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1f14eafc-2977-46fe-8cf6-4be2a18b5b21 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:49.762448 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1f14eafc-2977-46fe-8cf6-4be2a18b5b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:49.863403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 52/99] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:49.901660 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160ad0b-c30b-4001-beac-e7872e872336 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:49.941580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:50.041015 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d593885-7f86-406f-b753-d73bf56f62ee None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.041336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d593885-7f86-406f-b753-d73bf56f62ee None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.041588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d593885-7f86-406f-b753-d73bf56f62ee None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.043452 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 53/100] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:50 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:50.044530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde7b07-0749-48bb-a90c-48db08bc75fd None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.044817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde7b07-0749-48bb-a90c-48db08bc75fd None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.045099 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde7b07-0749-48bb-a90c-48db08bc75fd None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.046527 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 54/101] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:50 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:50.051900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ab20b-6440-40fd-aad3-bc033eec1fa1 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.052183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ab20b-6440-40fd-aad3-bc033eec1fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.052467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ab20b-6440-40fd-aad3-bc033eec1fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.087068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f26ab20b-6440-40fd-aad3-bc033eec1fa1 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:50.088546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f26ab20b-6440-40fd-aad3-bc033eec1fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:50.091320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5160ad0b-c30b-4001-beac-e7872e872336 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:50.161804 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd6220-86cc-46f4-ae18-ba7a927823e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:50.162384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd6220-86cc-46f4-ae18-ba7a927823e3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.162693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd6220-86cc-46f4-ae18-ba7a927823e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.162980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd6220-86cc-46f4-ae18-ba7a927823e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.187801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd6220-86cc-46f4-ae18-ba7a927823e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1ecd6220-86cc-46f4-ae18-ba7a927823e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:50.193737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:50.194630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:50.195439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:50.195439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:50.195439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:50.195439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:50.195439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:50.195439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:50.195439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:50.195439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:50.195439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:50.195439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:50.195439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:59:50.195439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 14:59:50.198438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 48/102] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 14:59:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:50.199984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2edbd-69ce-4420-8187-24af749ef820 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.200498 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2edbd-69ce-4420-8187-24af749ef820 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.200950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2edbd-69ce-4420-8187-24af749ef820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.220663 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-17a2edbd-69ce-4420-8187-24af749ef820 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:50.222352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-17a2edbd-69ce-4420-8187-24af749ef820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:50.232389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 55/103] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:50.233299 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ab20b-6440-40fd-aad3-bc033eec1fa1 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:50.318734 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 49/104] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:50.319889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2edbd-69ce-4420-8187-24af749ef820 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:50.341074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:50.359047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:50.396128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ab20b-6440-40fd-aad3-bc033eec1fa1 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:50.434583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae10fe-b2ef-4071-a33c-2162cdf375a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:50.435168 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae10fe-b2ef-4071-a33c-2162cdf375a1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.435515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae10fe-b2ef-4071-a33c-2162cdf375a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.435812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae10fe-b2ef-4071-a33c-2162cdf375a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.436540 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae10fe-b2ef-4071-a33c-2162cdf375a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:50.436854 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae10fe-b2ef-4071-a33c-2162cdf375a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:50.438259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae10fe-b2ef-4071-a33c-2162cdf375a1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:50.438530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2edbd-69ce-4420-8187-24af749ef820 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:50.447759 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 56/105] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 14:59:50 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:50.448747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef79c6-2260-4b82-b5e3-12dd3e690408 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.449043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef79c6-2260-4b82-b5e3-12dd3e690408 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.449325 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef79c6-2260-4b82-b5e3-12dd3e690408 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.450563 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 57/106] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:50.456308 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef79c6-2260-4b82-b5e3-12dd3e690408 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:50.480030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a30bf-9d9a-4177-97ee-df816f83d1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:50.480956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a30bf-9d9a-4177-97ee-df816f83d1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.481386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a30bf-9d9a-4177-97ee-df816f83d1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.481806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a30bf-9d9a-4177-97ee-df816f83d1e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.492996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a30bf-9d9a-4177-97ee-df816f83d1e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-ff3a30bf-9d9a-4177-97ee-df816f83d1e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:50.498817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:50.499790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:50.500580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:50.500580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:50.500580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:50.500580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:50.500580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:50.500580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:50.500580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:50.500580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:50.500580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:50.500580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:50.500580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:50.500580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 14:59:50.503115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 50/107] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 14:59:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:50.509170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4d513-2610-4dfc-91dc-b71f0379b997 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.509795 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4d513-2610-4dfc-91dc-b71f0379b997 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.510288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4d513-2610-4dfc-91dc-b71f0379b997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.530183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-64e4d513-2610-4dfc-91dc-b71f0379b997 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:50.531923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-64e4d513-2610-4dfc-91dc-b71f0379b997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:50.568823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef79c6-2260-4b82-b5e3-12dd3e690408 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:50.597825 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:50.609418 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5d9c1-be28-46d0-a225-d7ee5987ddda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:50.610071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5d9c1-be28-46d0-a225-d7ee5987ddda None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.610425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5d9c1-be28-46d0-a225-d7ee5987ddda None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.610784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5d9c1-be28-46d0-a225-d7ee5987ddda None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.612071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5d9c1-be28-46d0-a225-d7ee5987ddda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:50.613659 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5d9c1-be28-46d0-a225-d7ee5987ddda None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:50.615326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 58/108] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 14:59:50 2022] GET /identity/v3/domains/default => generated 200 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:50.616105 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5d9c1-be28-46d0-a225-d7ee5987ddda None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:50.617772 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 51/109] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:50.618866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4d513-2610-4dfc-91dc-b71f0379b997 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:50.627877 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:50.731998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4d513-2610-4dfc-91dc-b71f0379b997 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:50.740312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5d9c1-be28-46d0-a225-d7ee5987ddda None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:50.767578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9509b-9aec-4314-919f-a190347d65dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:50.768211 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9509b-9aec-4314-919f-a190347d65dc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.768496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9509b-9aec-4314-919f-a190347d65dc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.768777 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9509b-9aec-4314-919f-a190347d65dc None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.769509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9509b-9aec-4314-919f-a190347d65dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:50.769819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9509b-9aec-4314-919f-a190347d65dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:50.771260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9509b-9aec-4314-919f-a190347d65dc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:50.779237 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa556b-43ec-4a2b-9b64-deb9ae38c11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:50.779895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa556b-43ec-4a2b-9b64-deb9ae38c11f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.780183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa556b-43ec-4a2b-9b64-deb9ae38c11f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.780463 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa556b-43ec-4a2b-9b64-deb9ae38c11f None admin] PATH_INFO: `/v3/users/64cfd0d9528a405688a7d6185ba97242` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.780916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 52/110] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 14:59:50 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:50.781731 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9509b-9aec-4314-919f-a190347d65dc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:50.801945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa556b-43ec-4a2b-9b64-deb9ae38c11f None admin] RBAC: Authorizing `identity:get_user(user_id=64cfd0d9528a405688a7d6185ba97242)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:50.803501 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa556b-43ec-4a2b-9b64-deb9ae38c11f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:50.805266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 59/111] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:50 2022] GET /identity/v3/users/64cfd0d9528a405688a7d6185ba97242 => generated 290 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:50.806487 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879e194-900a-4087-86f7-9ca493044f9e None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.806824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879e194-900a-4087-86f7-9ca493044f9e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.807107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879e194-900a-4087-86f7-9ca493044f9e None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.810848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 60/112] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:50.810848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879e194-900a-4087-86f7-9ca493044f9e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:50.866972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:50.871871 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:50.906952 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9509b-9aec-4314-919f-a190347d65dc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:50.931924 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879e194-900a-4087-86f7-9ca493044f9e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:50.943654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fe1ac-0f7f-41bd-ab00-15029eb0c4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:50.944519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fe1ac-0f7f-41bd-ab00-15029eb0c4e5 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.944803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fe1ac-0f7f-41bd-ab00-15029eb0c4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.945081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fe1ac-0f7f-41bd-ab00-15029eb0c4e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.946313 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fe1ac-0f7f-41bd-ab00-15029eb0c4e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:50.947793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834fe1ac-0f7f-41bd-ab00-15029eb0c4e5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:50.971640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc0c94-f2ee-497c-8ef9-d3b4dcf4aebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:50.972602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc0c94-f2ee-497c-8ef9-d3b4dcf4aebf None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:50.972892 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc0c94-f2ee-497c-8ef9-d3b4dcf4aebf None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:50.973173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc0c94-f2ee-497c-8ef9-d3b4dcf4aebf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:50.976269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 53/113] 10.222.0.100 () {60 vars in 1146 bytes} [Mon Aug 1 14:59:50 2022] POST /identity/v3/projects => generated 303 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 14:59:50.987040 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc0c94-f2ee-497c-8ef9-d3b4dcf4aebf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:50.987719 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fe1ac-0f7f-41bd-ab00-15029eb0c4e5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-c7fc0c94-f2ee-497c-8ef9-d3b4dcf4aebf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:50.993944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:50.995213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:50.996019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:50.996019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:50.996019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:50.996019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:50.996019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:50.996019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:50.996019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:50.996019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:50.996019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:50.996019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:50.996019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:59:50.996019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 14:59:50.996019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 61/114] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 14:59:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:50.996698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc0c94-f2ee-497c-8ef9-d3b4dcf4aebf None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:51.123465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc0c94-f2ee-497c-8ef9-d3b4dcf4aebf None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:51.143541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:51.172041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fe1ac-0f7f-41bd-ab00-15029eb0c4e5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:51.179271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af4717-6e33-40fd-8897-7eb1332d0821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:51.179872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af4717-6e33-40fd-8897-7eb1332d0821 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:51.180198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af4717-6e33-40fd-8897-7eb1332d0821 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:51.181212 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af4717-6e33-40fd-8897-7eb1332d0821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:51.200719 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:51.224961 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00deeb69-2656-4d3e-a6b5-92c632460c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:51.225974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00deeb69-2656-4d3e-a6b5-92c632460c75 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:51.226462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00deeb69-2656-4d3e-a6b5-92c632460c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:51.227033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00deeb69-2656-4d3e-a6b5-92c632460c75 None admin] PATH_INFO: `/v3/users/24f06d52227a4059a53df8c2c849eefc` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:51.229010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00deeb69-2656-4d3e-a6b5-92c632460c75 None admin] RBAC: Authorizing `identity:get_user(user_id=24f06d52227a4059a53df8c2c849eefc)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:51.230770 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00deeb69-2656-4d3e-a6b5-92c632460c75 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:51.232959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 54/115] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:50 2022] GET /identity/v3/users/24f06d52227a4059a53df8c2c849eefc => generated 248 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:51.234522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58710e24-df74-4065-a738-a0d9cec686c7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:51.235168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58710e24-df74-4065-a738-a0d9cec686c7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:51.235740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58710e24-df74-4065-a738-a0d9cec686c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:51.254518 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-58710e24-df74-4065-a738-a0d9cec686c7 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:51.256079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-58710e24-df74-4065-a738-a0d9cec686c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:51.272895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af4717-6e33-40fd-8897-7eb1332d0821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:51.274974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af4717-6e33-40fd-8897-7eb1332d0821 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:51.340217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 55/116] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:51.347803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58710e24-df74-4065-a738-a0d9cec686c7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:51.390856 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 62/117] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 14:59:50 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:59:51.391732 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af4717-6e33-40fd-8897-7eb1332d0821 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:51.408887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:51.471936 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58710e24-df74-4065-a738-a0d9cec686c7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:51.495431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:51.512454 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af4717-6e33-40fd-8897-7eb1332d0821 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:51.525551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194f3a5-5d3a-4403-9e86-ba5820a7603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:51.526567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194f3a5-5d3a-4403-9e86-ba5820a7603f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:51.527122 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194f3a5-5d3a-4403-9e86-ba5820a7603f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:51.527538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194f3a5-5d3a-4403-9e86-ba5820a7603f None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:51.528781 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194f3a5-5d3a-4403-9e86-ba5820a7603f None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:51.529305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194f3a5-5d3a-4403-9e86-ba5820a7603f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:51.531829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194f3a5-5d3a-4403-9e86-ba5820a7603f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:51.551236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 56/118] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 14:59:51 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:51.551236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1e0a5-e4d5-47d6-8595-09da0c23f0e0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:51.551236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1e0a5-e4d5-47d6-8595-09da0c23f0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:51.551236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1e0a5-e4d5-47d6-8595-09da0c23f0e0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:51.552073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 57/119] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:51 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:51.553467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1e0a5-e4d5-47d6-8595-09da0c23f0e0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:51.581337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293df1d-6d68-4996-9517-bc4f72af4cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:51.582159 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293df1d-6d68-4996-9517-bc4f72af4cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:51.582577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293df1d-6d68-4996-9517-bc4f72af4cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:51.583065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293df1d-6d68-4996-9517-bc4f72af4cd4 None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:51.584680 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293df1d-6d68-4996-9517-bc4f72af4cd4 None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:51.587153 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293df1d-6d68-4996-9517-bc4f72af4cd4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:51.589981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 63/120] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 14:59:51 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:51.591045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293df1d-6d68-4996-9517-bc4f72af4cd4 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:51.680867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1e0a5-e4d5-47d6-8595-09da0c23f0e0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:51.687922 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:51.710627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293df1d-6d68-4996-9517-bc4f72af4cd4 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:51.719791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659f913-859b-4d61-9f61-676eb2ee68e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:51.720321 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659f913-859b-4d61-9f61-676eb2ee68e6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:51.720608 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659f913-859b-4d61-9f61-676eb2ee68e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:51.720889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659f913-859b-4d61-9f61-676eb2ee68e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:51.732172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659f913-859b-4d61-9f61-676eb2ee68e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0659f913-859b-4d61-9f61-676eb2ee68e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:51.737616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:51.738512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:51.744988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:51.744988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:51.744988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:51.744988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:51.744988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:51.744988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:51.744988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:51.744988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:51.744988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:51.744988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:51.744988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:59:51.744988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 14:59:51.744988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 58/121] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 14:59:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:51.744988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659f913-859b-4d61-9f61-676eb2ee68e6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:51.758866 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a8e64-9c93-4580-bc8c-450eecd20678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:51.759568 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a8e64-9c93-4580-bc8c-450eecd20678 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:51.759854 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a8e64-9c93-4580-bc8c-450eecd20678 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:51.760132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a8e64-9c93-4580-bc8c-450eecd20678 None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:51.761360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a8e64-9c93-4580-bc8c-450eecd20678 None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:51.762932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a8e64-9c93-4580-bc8c-450eecd20678 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:51.764568 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 64/122] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 14:59:51 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:51.768594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866574c5-c2f6-4691-81e9-a6033691de6c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:51.768965 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866574c5-c2f6-4691-81e9-a6033691de6c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:51.769317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866574c5-c2f6-4691-81e9-a6033691de6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:51.792486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:51.793814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-866574c5-c2f6-4691-81e9-a6033691de6c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:51.795147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-866574c5-c2f6-4691-81e9-a6033691de6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:51.868492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659f913-859b-4d61-9f61-676eb2ee68e6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:51.886691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 65/123] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:51.886691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866574c5-c2f6-4691-81e9-a6033691de6c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:51.921376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1143387-dddc-4511-9729-9b1629953c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:51.921376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1143387-dddc-4511-9729-9b1629953c67 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:51.921376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1143387-dddc-4511-9729-9b1629953c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:51.921376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1143387-dddc-4511-9729-9b1629953c67 None admin] PATH_INFO: `/v3/users/9d8a17c4135b4de4bf1d369d28f1c797` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:51.922397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1143387-dddc-4511-9729-9b1629953c67 None admin] RBAC: Authorizing `identity:get_user(user_id=9d8a17c4135b4de4bf1d369d28f1c797)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:51.924066 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1143387-dddc-4511-9729-9b1629953c67 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:51.926056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 59/124] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:51 2022] GET /identity/v3/users/9d8a17c4135b4de4bf1d369d28f1c797 => generated 276 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:51.927589 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1143387-dddc-4511-9729-9b1629953c67 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:51.984804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:52.010875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866574c5-c2f6-4691-81e9-a6033691de6c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:52.041185 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1143387-dddc-4511-9729-9b1629953c67 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:52.046884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46370d09-f42d-4fbd-bcaa-f01c393e51fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:52.047688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46370d09-f42d-4fbd-bcaa-f01c393e51fd None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:52.048486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46370d09-f42d-4fbd-bcaa-f01c393e51fd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:52.048869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46370d09-f42d-4fbd-bcaa-f01c393e51fd None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80/users/24f06d52227a4059a53df8c2c849eefc/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:52.050301 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:52.051971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46370d09-f42d-4fbd-bcaa-f01c393e51fd None admin] RBAC: Authorizing `identity:create_grant(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80, user_id=24f06d52227a4059a53df8c2c849eefc, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:52.054247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46370d09-f42d-4fbd-bcaa-f01c393e51fd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:52.067614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 66/125] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 14:59:51 2022] PUT /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80/users/24f06d52227a4059a53df8c2c849eefc/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:59:52.068675 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46370d09-f42d-4fbd-bcaa-f01c393e51fd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:52.091037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb4520-d8d3-4eda-8f68-536914a64374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:52.091684 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb4520-d8d3-4eda-8f68-536914a64374 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:52.091972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb4520-d8d3-4eda-8f68-536914a64374 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:52.092250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb4520-d8d3-4eda-8f68-536914a64374 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:52.093066 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb4520-d8d3-4eda-8f68-536914a64374 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:52.093590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb4520-d8d3-4eda-8f68-536914a64374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:52.095299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb4520-d8d3-4eda-8f68-536914a64374 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:52.105861 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 60/126] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 14:59:51 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:52.106655 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb4520-d8d3-4eda-8f68-536914a64374 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:52.220150 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46370d09-f42d-4fbd-bcaa-f01c393e51fd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:52.249139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb4520-d8d3-4eda-8f68-536914a64374 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:52.270735 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb2ca6-aed2-41f5-ab33-91db7abd1a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:52.271374 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb2ca6-aed2-41f5-ab33-91db7abd1a3e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:52.271650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb2ca6-aed2-41f5-ab33-91db7abd1a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:52.271943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb2ca6-aed2-41f5-ab33-91db7abd1a3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:52.272729 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb2ca6-aed2-41f5-ab33-91db7abd1a3e None admin] RBAC: Adding query filter params (role.id=471fd358281343cba97902b70b416f32, scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80, user.id=64cfd0d9528a405688a7d6185ba97242) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:52.273040 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb2ca6-aed2-41f5-ab33-91db7abd1a3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:52.275139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb2ca6-aed2-41f5-ab33-91db7abd1a3e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:52.284422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 67/127] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 14:59:52 2022] GET /identity/v3/role_assignments?user.id=64cfd0d9528a405688a7d6185ba97242&scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80&role.id=471fd358281343cba97902b70b416f32 => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:52.285486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb2ca6-aed2-41f5-ab33-91db7abd1a3e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:52.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:52.296664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc159b57-f24a-4a7b-a922-9417ff8ad289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:52.297594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc159b57-f24a-4a7b-a922-9417ff8ad289 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:52.298208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc159b57-f24a-4a7b-a922-9417ff8ad289 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:52.298645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc159b57-f24a-4a7b-a922-9417ff8ad289 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:52.316199 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc159b57-f24a-4a7b-a922-9417ff8ad289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-fc159b57-f24a-4a7b-a922-9417ff8ad289 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:52.324514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:52.325465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:52.326240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:52.326240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:52.326240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:52.326240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:52.326240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:52.326240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:52.326240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:52.326240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:52.326240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:52.326240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:52.326240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:59:52.326240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 14:59:52.328099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 61/128] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 14:59:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:52.329096 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc159b57-f24a-4a7b-a922-9417ff8ad289 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:52.366773 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:52.449941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb2ca6-aed2-41f5-ab33-91db7abd1a3e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:52.452430 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc159b57-f24a-4a7b-a922-9417ff8ad289 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:52.490929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cf52a-5106-4852-b9ab-627dbee7d13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:52.491939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cf52a-5106-4852-b9ab-627dbee7d13b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:52.492592 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cf52a-5106-4852-b9ab-627dbee7d13b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:52.493612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4cf52a-5106-4852-b9ab-627dbee7d13b None admin] PATH_INFO: `/v3/users/9d8a17c4135b4de4bf1d369d28f1c797` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:52.495219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cf52a-5106-4852-b9ab-627dbee7d13b None admin] RBAC: Authorizing `identity:get_user(user_id=9d8a17c4135b4de4bf1d369d28f1c797)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:52.497011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4cf52a-5106-4852-b9ab-627dbee7d13b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:52.498964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 62/129] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:52 2022] GET /identity/v3/users/9d8a17c4135b4de4bf1d369d28f1c797 => generated 276 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:52.500158 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cf52a-5106-4852-b9ab-627dbee7d13b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:52.502525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf71347-a5db-4ab3-9f10-05389e33a5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:52.503228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf71347-a5db-4ab3-9f10-05389e33a5bf None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:52.503519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf71347-a5db-4ab3-9f10-05389e33a5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:52.503799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf71347-a5db-4ab3-9f10-05389e33a5bf None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:52.514722 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf71347-a5db-4ab3-9f10-05389e33a5bf None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:52.514722 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf71347-a5db-4ab3-9f10-05389e33a5bf None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:52.514722 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 68/130] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 14:59:52 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:52.515042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf71347-a5db-4ab3-9f10-05389e33a5bf None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:52.623547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:52.664151 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf71347-a5db-4ab3-9f10-05389e33a5bf None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:52.667401 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:52.676966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4cf52a-5106-4852-b9ab-627dbee7d13b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:52.712632 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1636a76-66e2-4ed2-b7b3-19583eeca796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:52.713549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1636a76-66e2-4ed2-b7b3-19583eeca796 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:52.713838 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1636a76-66e2-4ed2-b7b3-19583eeca796 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:52.714118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1636a76-66e2-4ed2-b7b3-19583eeca796 None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:52.716253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1636a76-66e2-4ed2-b7b3-19583eeca796 None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:52.717743 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1636a76-66e2-4ed2-b7b3-19583eeca796 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:52.720966 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 69/131] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 14:59:52 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:52.723982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1636a76-66e2-4ed2-b7b3-19583eeca796 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:52.755162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a60eba-7a14-4d4b-be15-e2bc3ede4cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:52.755885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a60eba-7a14-4d4b-be15-e2bc3ede4cee None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:52.756072 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a60eba-7a14-4d4b-be15-e2bc3ede4cee None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:52.756358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a60eba-7a14-4d4b-be15-e2bc3ede4cee None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:52.757092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a60eba-7a14-4d4b-be15-e2bc3ede4cee None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:52.757409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a60eba-7a14-4d4b-be15-e2bc3ede4cee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:52.758858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a60eba-7a14-4d4b-be15-e2bc3ede4cee None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:52.776703 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 63/132] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 14:59:52 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:52.777533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a60eba-7a14-4d4b-be15-e2bc3ede4cee None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:52.908924 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1636a76-66e2-4ed2-b7b3-19583eeca796 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:52.971540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a60eba-7a14-4d4b-be15-e2bc3ede4cee None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:52.974355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b6f56-ffa6-4b05-a9a6-6d5e3cb6d01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:52.975084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b6f56-ffa6-4b05-a9a6-6d5e3cb6d01e None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:52.975373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b6f56-ffa6-4b05-a9a6-6d5e3cb6d01e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:52.975657 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b6f56-ffa6-4b05-a9a6-6d5e3cb6d01e None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80/users/9d8a17c4135b4de4bf1d369d28f1c797/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:52.979361 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b6f56-ffa6-4b05-a9a6-6d5e3cb6d01e None admin] RBAC: Authorizing `identity:create_grant(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80, user_id=9d8a17c4135b4de4bf1d369d28f1c797, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:52.984947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b6f56-ffa6-4b05-a9a6-6d5e3cb6d01e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:53.000426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:53.006807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 70/133] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 14:59:52 2022] PUT /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80/users/9d8a17c4135b4de4bf1d369d28f1c797/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:59:53.007464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b6f56-ffa6-4b05-a9a6-6d5e3cb6d01e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:53.045857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90a20f-4464-4bbe-b74e-a80ebfff1407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:53.046567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90a20f-4464-4bbe-b74e-a80ebfff1407 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:53.046885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90a20f-4464-4bbe-b74e-a80ebfff1407 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:53.047171 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90a20f-4464-4bbe-b74e-a80ebfff1407 None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80/users/9d8a17c4135b4de4bf1d369d28f1c797/roles/57221d717a454c7fae4f2df1de9dbe5b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:53.053196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b90a20f-4464-4bbe-b74e-a80ebfff1407 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80, user_id=9d8a17c4135b4de4bf1d369d28f1c797, role_id=57221d717a454c7fae4f2df1de9dbe5b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:53.057775 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b90a20f-4464-4bbe-b74e-a80ebfff1407 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:53.078499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 64/134] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 14:59:52 2022] PUT /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80/users/9d8a17c4135b4de4bf1d369d28f1c797/roles/57221d717a454c7fae4f2df1de9dbe5b => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 14:59:53.193998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b6f56-ffa6-4b05-a9a6-6d5e3cb6d01e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:53.225541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ce2f3-31aa-48d3-a44a-f4fcc8fc4b75 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:53.225855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ce2f3-31aa-48d3-a44a-f4fcc8fc4b75 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:53.226118 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7ce2f3-31aa-48d3-a44a-f4fcc8fc4b75 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:53.227650 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 65/135] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 14:59:53.233955 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1cfe5-08c9-4ae3-987e-2abc8b8e5ddb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:53.234243 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1cfe5-08c9-4ae3-987e-2abc8b8e5ddb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:53.234525 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1cfe5-08c9-4ae3-987e-2abc8b8e5ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:53.265416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-68f1cfe5-08c9-4ae3-987e-2abc8b8e5ddb None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:53.266682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-68f1cfe5-08c9-4ae3-987e-2abc8b8e5ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:53.272567 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31f02d-ace9-4832-94a8-6dac71d6d292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:53.273249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31f02d-ace9-4832-94a8-6dac71d6d292 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:53.273540 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31f02d-ace9-4832-94a8-6dac71d6d292 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:53.274376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31f02d-ace9-4832-94a8-6dac71d6d292 None admin] PATH_INFO: `/v3/users/9d8a17c4135b4de4bf1d369d28f1c797` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:53.276418 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31f02d-ace9-4832-94a8-6dac71d6d292 None admin] RBAC: Authorizing `identity:get_user(user_id=9d8a17c4135b4de4bf1d369d28f1c797)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:53.277915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31f02d-ace9-4832-94a8-6dac71d6d292 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:53.284669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 71/136] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:53 2022] GET /identity/v3/users/9d8a17c4135b4de4bf1d369d28f1c797 => generated 276 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:53.290957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31f02d-ace9-4832-94a8-6dac71d6d292 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:53.303322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:53.393148 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 66/137] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:59:53.406222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:53.421407 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1cfe5-08c9-4ae3-987e-2abc8b8e5ddb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:53.443553 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31f02d-ace9-4832-94a8-6dac71d6d292 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:53.485687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b934f-02e6-48ec-9f38-7b937b533377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:53.486325 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b934f-02e6-48ec-9f38-7b937b533377 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:53.486620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b934f-02e6-48ec-9f38-7b937b533377 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:53.486961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b934f-02e6-48ec-9f38-7b937b533377 None admin] PATH_INFO: `/v3/projects/0ab4610904d94edd8eb3a1f420c13314` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:53.488159 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b934f-02e6-48ec-9f38-7b937b533377 None admin] RBAC: Authorizing `identity:get_project(project_id=0ab4610904d94edd8eb3a1f420c13314)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:53.489742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b934f-02e6-48ec-9f38-7b937b533377 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:53.491450 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 72/138] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 14:59:53 2022] GET /identity/v3/projects/0ab4610904d94edd8eb3a1f420c13314 => generated 313 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:53.497667 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b934f-02e6-48ec-9f38-7b937b533377 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:53.533900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1cfe5-08c9-4ae3-987e-2abc8b8e5ddb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:53.575489 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be2cc3-cbe3-43ce-93ff-40b019a4ec13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:53.576207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be2cc3-cbe3-43ce-93ff-40b019a4ec13 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:53.576491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be2cc3-cbe3-43ce-93ff-40b019a4ec13 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:53.576870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be2cc3-cbe3-43ce-93ff-40b019a4ec13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:53.583316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be2cc3-cbe3-43ce-93ff-40b019a4ec13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:53.584954 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be2cc3-cbe3-43ce-93ff-40b019a4ec13 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:53.586626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 67/139] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 14:59:53 2022] GET /identity/v3/domains/default => generated 200 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:53.592741 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be2cc3-cbe3-43ce-93ff-40b019a4ec13 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:53.624396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b934f-02e6-48ec-9f38-7b937b533377 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:53.645219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:53.661712 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c350e7-8a1d-4e21-92f8-07d4c8e7640c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:53.662646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c350e7-8a1d-4e21-92f8-07d4c8e7640c None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:53.663127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c350e7-8a1d-4e21-92f8-07d4c8e7640c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:53.663552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c350e7-8a1d-4e21-92f8-07d4c8e7640c None admin] PATH_INFO: `/v3/projects/0ab4610904d94edd8eb3a1f420c13314/users/9d8a17c4135b4de4bf1d369d28f1c797/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:53.665940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c350e7-8a1d-4e21-92f8-07d4c8e7640c None admin] RBAC: Authorizing `identity:create_grant(project_id=0ab4610904d94edd8eb3a1f420c13314, user_id=9d8a17c4135b4de4bf1d369d28f1c797, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:53.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c350e7-8a1d-4e21-92f8-07d4c8e7640c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:53.672121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:53.691540 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 73/140] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 14:59:53 2022] PUT /identity/v3/projects/0ab4610904d94edd8eb3a1f420c13314/users/9d8a17c4135b4de4bf1d369d28f1c797/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:59:53.751483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be2cc3-cbe3-43ce-93ff-40b019a4ec13 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:53.798567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6b522-0fe6-4336-8488-e9ba7ac7f59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:53.799366 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6b522-0fe6-4336-8488-e9ba7ac7f59c None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:53.799529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6b522-0fe6-4336-8488-e9ba7ac7f59c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:53.799827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6b522-0fe6-4336-8488-e9ba7ac7f59c None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:53.800535 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6b522-0fe6-4336-8488-e9ba7ac7f59c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:53.802013 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6b522-0fe6-4336-8488-e9ba7ac7f59c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:53.842076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 68/141] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 14:59:53 2022] POST /identity/v3/users => generated 284 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 14:59:54.868595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac64718-1112-450b-914c-d05bc35c3342 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:54.869662 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac64718-1112-450b-914c-d05bc35c3342 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:54.870167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac64718-1112-450b-914c-d05bc35c3342 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:54.871820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 74/142] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:54 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:54.877373 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd335656-b287-474e-8b0f-4a00daefccec None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:54.878276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd335656-b287-474e-8b0f-4a00daefccec None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:54.878797 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd335656-b287-474e-8b0f-4a00daefccec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:54.900794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fd335656-b287-474e-8b0f-4a00daefccec None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:54.902360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fd335656-b287-474e-8b0f-4a00daefccec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:54.944239 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162c0c8-01b0-41b8-8d5e-13f0ffd27146 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:54.944572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162c0c8-01b0-41b8-8d5e-13f0ffd27146 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:54.944814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162c0c8-01b0-41b8-8d5e-13f0ffd27146 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:54.946940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 75/143] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:54 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:54.952253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc000adb-bbbf-4a27-a4a3-45e259327943 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:54.952547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc000adb-bbbf-4a27-a4a3-45e259327943 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:54.952834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc000adb-bbbf-4a27-a4a3-45e259327943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:54.987716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-fc000adb-bbbf-4a27-a4a3-45e259327943 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:54.988945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fc000adb-bbbf-4a27-a4a3-45e259327943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:55.031648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 69/144] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:55.081093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd335656-b287-474e-8b0f-4a00daefccec None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:55.087795 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:55.102510 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 76/145] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:59:55.155314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc000adb-bbbf-4a27-a4a3-45e259327943 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:55.272207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd335656-b287-474e-8b0f-4a00daefccec None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:55.299892 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc000adb-bbbf-4a27-a4a3-45e259327943 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:55.313441 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffe637-3ecb-45ab-b60e-6bba5f99c9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:55.314255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffe637-3ecb-45ab-b60e-6bba5f99c9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:55.314604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffe637-3ecb-45ab-b60e-6bba5f99c9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:55.314979 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffe637-3ecb-45ab-b60e-6bba5f99c9a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:55.331158 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:55.337570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffe637-3ecb-45ab-b60e-6bba5f99c9a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9dffe637-3ecb-45ab-b60e-6bba5f99c9a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:55.345983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:55.346946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:55.347865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:55.347865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:55.347865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:55.347865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:55.347865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:55.347865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:55.347865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:55.347865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:55.347865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:55.347865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:55.347865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 14:59:55.347865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 14:59:55.347865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 70/146] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 14:59:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:55.348542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc9e21-590c-4e44-8fd8-6fcc55d3c18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:55.348542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc9e21-590c-4e44-8fd8-6fcc55d3c18c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:55.348542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc9e21-590c-4e44-8fd8-6fcc55d3c18c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:55.348755 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc9e21-590c-4e44-8fd8-6fcc55d3c18c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:55.355145 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffe637-3ecb-45ab-b60e-6bba5f99c9a7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:55.364145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc9e21-590c-4e44-8fd8-6fcc55d3c18c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-c6fc9e21-590c-4e44-8fd8-6fcc55d3c18c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:55.369816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:55.370697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:55.371736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:55.371736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:55.371736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:55.371736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:55.371736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:55.371736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:55.371736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:55.371736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:55.371736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:55.371736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:55.371736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 14:59:55.371736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 14:59:55.371736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 77/147] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 14:59:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:55.376692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc9e21-590c-4e44-8fd8-6fcc55d3c18c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:55.460234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:55.492925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc9e21-590c-4e44-8fd8-6fcc55d3c18c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:55.528145 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffe637-3ecb-45ab-b60e-6bba5f99c9a7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:55.530975 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7feeec0-df4d-4e24-8ef8-5ae3e190869f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:55.531621 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7feeec0-df4d-4e24-8ef8-5ae3e190869f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:55.531897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7feeec0-df4d-4e24-8ef8-5ae3e190869f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:55.532176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7feeec0-df4d-4e24-8ef8-5ae3e190869f None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:55.532918 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7feeec0-df4d-4e24-8ef8-5ae3e190869f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:55.533222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7feeec0-df4d-4e24-8ef8-5ae3e190869f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:55.534647 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7feeec0-df4d-4e24-8ef8-5ae3e190869f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:55.544213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 78/148] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 14:59:55 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:55.555880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7feeec0-df4d-4e24-8ef8-5ae3e190869f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:55.595048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17642a-72d6-4ef4-8ead-9e94a88c6a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:55.596107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17642a-72d6-4ef4-8ead-9e94a88c6a25 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:55.596741 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17642a-72d6-4ef4-8ead-9e94a88c6a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:55.597292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17642a-72d6-4ef4-8ead-9e94a88c6a25 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:55.598276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17642a-72d6-4ef4-8ead-9e94a88c6a25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:55.598839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17642a-72d6-4ef4-8ead-9e94a88c6a25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:55.600861 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17642a-72d6-4ef4-8ead-9e94a88c6a25 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:55.616028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 71/149] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 14:59:55 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:55.621389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:55.623871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17642a-72d6-4ef4-8ead-9e94a88c6a25 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:55.693538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7feeec0-df4d-4e24-8ef8-5ae3e190869f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:55.744721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d7a62-7f13-4087-a94a-f134b79718fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:55.745320 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d7a62-7f13-4087-a94a-f134b79718fd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:55.745605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d7a62-7f13-4087-a94a-f134b79718fd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:55.745882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d7a62-7f13-4087-a94a-f134b79718fd None admin] PATH_INFO: `/v3/users/64cfd0d9528a405688a7d6185ba97242` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:55.747132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d7a62-7f13-4087-a94a-f134b79718fd None admin] RBAC: Authorizing `identity:get_user(user_id=64cfd0d9528a405688a7d6185ba97242)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:55.748616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d7a62-7f13-4087-a94a-f134b79718fd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:55.750468 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 79/150] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:55 2022] GET /identity/v3/users/64cfd0d9528a405688a7d6185ba97242 => generated 290 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:55.755794 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d7a62-7f13-4087-a94a-f134b79718fd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:55.794487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17642a-72d6-4ef4-8ead-9e94a88c6a25 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:55.841326 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:55.856545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970e460-42a1-4658-aeb7-29b19844f2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:55.857992 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970e460-42a1-4658-aeb7-29b19844f2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:55.858592 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970e460-42a1-4658-aeb7-29b19844f2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:55.859228 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970e460-42a1-4658-aeb7-29b19844f2a4 None admin] PATH_INFO: `/v3/users/24f06d52227a4059a53df8c2c849eefc` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:55.860668 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970e460-42a1-4658-aeb7-29b19844f2a4 None admin] RBAC: Authorizing `identity:get_user(user_id=24f06d52227a4059a53df8c2c849eefc)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:55.862507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970e460-42a1-4658-aeb7-29b19844f2a4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:55.864984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 72/151] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:55 2022] GET /identity/v3/users/24f06d52227a4059a53df8c2c849eefc => generated 248 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:55.871658 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970e460-42a1-4658-aeb7-29b19844f2a4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:55.935589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d7a62-7f13-4087-a94a-f134b79718fd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:55.968442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:56.020410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a35c0-81e7-4a28-8302-53136b5f9394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:56.021000 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a35c0-81e7-4a28-8302-53136b5f9394 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:56.021290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a35c0-81e7-4a28-8302-53136b5f9394 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:56.021570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a35c0-81e7-4a28-8302-53136b5f9394 None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:56.022865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a35c0-81e7-4a28-8302-53136b5f9394 None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:56.024425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a35c0-81e7-4a28-8302-53136b5f9394 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:56.026176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 80/152] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 14:59:55 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:56.031910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a35c0-81e7-4a28-8302-53136b5f9394 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:56.051102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970e460-42a1-4658-aeb7-29b19844f2a4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:56.118289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933f855-d3d2-42a0-9365-15278497b62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:56.119860 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933f855-d3d2-42a0-9365-15278497b62f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:56.120761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933f855-d3d2-42a0-9365-15278497b62f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:56.121256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933f855-d3d2-42a0-9365-15278497b62f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:56.226249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:56.235207 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a35c0-81e7-4a28-8302-53136b5f9394 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:56.246781 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933f855-d3d2-42a0-9365-15278497b62f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:56.248697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933f855-d3d2-42a0-9365-15278497b62f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:56.307265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a697e1d-5f79-4c1c-8a96-345f3be92259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:56.307881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a697e1d-5f79-4c1c-8a96-345f3be92259 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:56.308168 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a697e1d-5f79-4c1c-8a96-345f3be92259 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:56.308450 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a697e1d-5f79-4c1c-8a96-345f3be92259 None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80/users/64cfd0d9528a405688a7d6185ba97242/roles/471fd358281343cba97902b70b416f32` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:56.316102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a697e1d-5f79-4c1c-8a96-345f3be92259 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80, user_id=64cfd0d9528a405688a7d6185ba97242, role_id=471fd358281343cba97902b70b416f32)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:56.317580 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a697e1d-5f79-4c1c-8a96-345f3be92259 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:56.341531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 81/153] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 14:59:56 2022] PUT /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80/users/64cfd0d9528a405688a7d6185ba97242/roles/471fd358281343cba97902b70b416f32 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 14:59:56.402683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 73/154] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 14:59:55 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:59:56.408912 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a697e1d-5f79-4c1c-8a96-345f3be92259 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:56.467527 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:56.533888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a697e1d-5f79-4c1c-8a96-345f3be92259 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:56.575687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf16e1-ba74-4cbb-8858-671871c28f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:56.576295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf16e1-ba74-4cbb-8858-671871c28f1d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:56.576586 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf16e1-ba74-4cbb-8858-671871c28f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:56.576924 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf16e1-ba74-4cbb-8858-671871c28f1d None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:56.580318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf16e1-ba74-4cbb-8858-671871c28f1d None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:56.581951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf16e1-ba74-4cbb-8858-671871c28f1d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:56.583981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 82/155] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 14:59:56 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:56.590947 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933f855-d3d2-42a0-9365-15278497b62f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:56.745424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:56.754398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933f855-d3d2-42a0-9365-15278497b62f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:56.775977 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fe09b-4bc8-4888-903b-2d7916fb23d6 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:56.776346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fe09b-4bc8-4888-903b-2d7916fb23d6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:56.776624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fe09b-4bc8-4888-903b-2d7916fb23d6 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:56.778027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 83/156] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:56.783513 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d31e40-9660-43fc-a2a4-1dec325ee6f4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:56.783800 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d31e40-9660-43fc-a2a4-1dec325ee6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:56.784084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d31e40-9660-43fc-a2a4-1dec325ee6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:56.798597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fac0a-c13e-40bb-94e7-c0479a1fcf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:56.799615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fac0a-c13e-40bb-94e7-c0479a1fcf17 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:56.800106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fac0a-c13e-40bb-94e7-c0479a1fcf17 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:56.800691 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fac0a-c13e-40bb-94e7-c0479a1fcf17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:56.801974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fac0a-c13e-40bb-94e7-c0479a1fcf17 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80, user.id=24f06d52227a4059a53df8c2c849eefc) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:56.802558 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fac0a-c13e-40bb-94e7-c0479a1fcf17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:56.804249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fac0a-c13e-40bb-94e7-c0479a1fcf17 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:56.811530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 74/157] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 14:59:56 2022] GET /identity/v3/role_assignments?user.id=24f06d52227a4059a53df8c2c849eefc&scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 610 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:56.824941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a7d31e40-9660-43fc-a2a4-1dec325ee6f4 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:56.826248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a7d31e40-9660-43fc-a2a4-1dec325ee6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:56.923576 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 84/158] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 14:59:56.974328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fac0a-c13e-40bb-94e7-c0479a1fcf17 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:57.127467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fac0a-c13e-40bb-94e7-c0479a1fcf17 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:57.163456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81b114-2f5a-41fc-9ad5-d0815e79d601 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:57.163995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81b114-2f5a-41fc-9ad5-d0815e79d601 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:57.164072 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81b114-2f5a-41fc-9ad5-d0815e79d601 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:57.165968 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 85/159] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:57 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 14:59:57.171961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c70aa9-c703-4d01-b38d-1812ef2a215f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:57.172241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c70aa9-c703-4d01-b38d-1812ef2a215f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:57.172512 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c70aa9-c703-4d01-b38d-1812ef2a215f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:57.179439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a413d-554c-476d-ae2a-ac4d79612663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:57.181012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a413d-554c-476d-ae2a-ac4d79612663 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:57.181499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a413d-554c-476d-ae2a-ac4d79612663 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:57.181990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678a413d-554c-476d-ae2a-ac4d79612663 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:57.196803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678a413d-554c-476d-ae2a-ac4d79612663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-678a413d-554c-476d-ae2a-ac4d79612663 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:57.202202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:57.203103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:57.203892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:57.203892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:57.203892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:57.203892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:57.203892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:57.203892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:57.203892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:57.203892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:57.203892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:57.203892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:57.203892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:59:57.203892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 14:59:57.203892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 75/160] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 14:59:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:57.207725 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-97c70aa9-c703-4d01-b38d-1812ef2a215f None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:57.208922 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-97c70aa9-c703-4d01-b38d-1812ef2a215f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:57.212230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a413d-554c-476d-ae2a-ac4d79612663 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:57.251711 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:57.290555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:57.378113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 86/161] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 01 14:59:57.442173 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678a413d-554c-476d-ae2a-ac4d79612663 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:57.478728 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c70aa9-c703-4d01-b38d-1812ef2a215f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:57.537687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e73dc-2875-4e3c-a6ba-9b8deb704e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:57.537687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e73dc-2875-4e3c-a6ba-9b8deb704e82 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:57.537687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e73dc-2875-4e3c-a6ba-9b8deb704e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:57.537687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e73dc-2875-4e3c-a6ba-9b8deb704e82 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:57.538149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e73dc-2875-4e3c-a6ba-9b8deb704e82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:57.538149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e73dc-2875-4e3c-a6ba-9b8deb704e82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:57.539023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e73dc-2875-4e3c-a6ba-9b8deb704e82 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:57.559894 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 76/162] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 14:59:57 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:57.567452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e73dc-2875-4e3c-a6ba-9b8deb704e82 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:57.684700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c70aa9-c703-4d01-b38d-1812ef2a215f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:57.727553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:57.761140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e73dc-2875-4e3c-a6ba-9b8deb704e82 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:57.773439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c95ef4-acee-417f-aeb5-cbc05a261f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:57.774654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c95ef4-acee-417f-aeb5-cbc05a261f17 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:57.774953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c95ef4-acee-417f-aeb5-cbc05a261f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:57.775223 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c95ef4-acee-417f-aeb5-cbc05a261f17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:57.795293 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c95ef4-acee-417f-aeb5-cbc05a261f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-83c95ef4-acee-417f-aeb5-cbc05a261f17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:57.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:57.801297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:57.802076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:57.802076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:57.802076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:57.802076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:57.802076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:57.802076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:57.802076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:57.802076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:57.802076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:57.802076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:57.802076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:59:57.802076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 14:59:57.802076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 87/163] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 14:59:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 14:59:57.811329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c95ef4-acee-417f-aeb5-cbc05a261f17 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:57.835421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03caf5ba-cd0b-47d4-b868-9b69a359a7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:57.836181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03caf5ba-cd0b-47d4-b868-9b69a359a7bf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:57.836984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03caf5ba-cd0b-47d4-b868-9b69a359a7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:57.837268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03caf5ba-cd0b-47d4-b868-9b69a359a7bf None admin] PATH_INFO: `/v3/users/9d8a17c4135b4de4bf1d369d28f1c797` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:57.838819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03caf5ba-cd0b-47d4-b868-9b69a359a7bf None admin] RBAC: Authorizing `identity:get_user(user_id=9d8a17c4135b4de4bf1d369d28f1c797)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:57.840302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03caf5ba-cd0b-47d4-b868-9b69a359a7bf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:57.841939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 77/164] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:57 2022] GET /identity/v3/users/9d8a17c4135b4de4bf1d369d28f1c797 => generated 276 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:57.850710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03caf5ba-cd0b-47d4-b868-9b69a359a7bf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:57.863611 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:57.993914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c95ef4-acee-417f-aeb5-cbc05a261f17 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:58.062444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f9367-c6b6-4407-84e2-6785855eb3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:58.063178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f9367-c6b6-4407-84e2-6785855eb3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:58.063854 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f9367-c6b6-4407-84e2-6785855eb3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:58.064146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f9367-c6b6-4407-84e2-6785855eb3d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:58.065020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f9367-c6b6-4407-84e2-6785855eb3d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:58.065329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f9367-c6b6-4407-84e2-6785855eb3d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:58.066738 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f9367-c6b6-4407-84e2-6785855eb3d8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:58.083012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 88/165] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 14:59:57 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:58.090627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f9367-c6b6-4407-84e2-6785855eb3d8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:58.108994 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03caf5ba-cd0b-47d4-b868-9b69a359a7bf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:58.183106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61a696-64b7-488b-8bac-9de31907630f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:58.183765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61a696-64b7-488b-8bac-9de31907630f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:58.184014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61a696-64b7-488b-8bac-9de31907630f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:58.184293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61a696-64b7-488b-8bac-9de31907630f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:58.228764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:58.303841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:58.324601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f9367-c6b6-4407-84e2-6785855eb3d8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:58.330829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61a696-64b7-488b-8bac-9de31907630f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:58.337229 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61a696-64b7-488b-8bac-9de31907630f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:58.395936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b15fc-7cdb-4d67-b4cc-e86879181092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:58.397083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b15fc-7cdb-4d67-b4cc-e86879181092 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:58.398400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b15fc-7cdb-4d67-b4cc-e86879181092 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:58.399504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b15fc-7cdb-4d67-b4cc-e86879181092 None admin] PATH_INFO: `/v3/users/9d8a17c4135b4de4bf1d369d28f1c797` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:58.402890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b15fc-7cdb-4d67-b4cc-e86879181092 None admin] RBAC: Authorizing `identity:get_user(user_id=9d8a17c4135b4de4bf1d369d28f1c797)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:58.404644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009b15fc-7cdb-4d67-b4cc-e86879181092 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:58.412250 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 89/166] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:58 2022] GET /identity/v3/users/9d8a17c4135b4de4bf1d369d28f1c797 => generated 276 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 14:59:58.413918 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48b367-441e-4be2-bd39-75de547ea573 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:58.415157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48b367-441e-4be2-bd39-75de547ea573 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:58.415653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48b367-441e-4be2-bd39-75de547ea573 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:58.417710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 90/167] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:58 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:58.420123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48b367-441e-4be2-bd39-75de547ea573 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:58.531140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 78/168] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 14:59:57 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 681 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 14:59:58.532842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e07e9-abb3-4cea-ad2c-6e27e0cff89d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:58.533553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e07e9-abb3-4cea-ad2c-6e27e0cff89d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:58.534079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36e07e9-abb3-4cea-ad2c-6e27e0cff89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:58.571034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48b367-441e-4be2-bd39-75de547ea573 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:58.585823 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d36e07e9-abb3-4cea-ad2c-6e27e0cff89d None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:58.587713 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d36e07e9-abb3-4cea-ad2c-6e27e0cff89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:58.609670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60ef09-eff9-48a1-a317-a009531b2260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:58.610479 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60ef09-eff9-48a1-a317-a009531b2260 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:58.610793 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60ef09-eff9-48a1-a317-a009531b2260 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:58.611068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60ef09-eff9-48a1-a317-a009531b2260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:58.645238 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:58.666898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:58.695887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60ef09-eff9-48a1-a317-a009531b2260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:58.697889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60ef09-eff9-48a1-a317-a009531b2260 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:58.716167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 79/169] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:58.716167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e07e9-abb3-4cea-ad2c-6e27e0cff89d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:58.847870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 91/170] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 14:59:58 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 14:59:58.849301 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60ef09-eff9-48a1-a317-a009531b2260 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:58.944632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36e07e9-abb3-4cea-ad2c-6e27e0cff89d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:58.956021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:59.031399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec712896-76e3-4a2a-a783-4aed7b9522e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:59.032119 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec712896-76e3-4a2a-a783-4aed7b9522e7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.032409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec712896-76e3-4a2a-a783-4aed7b9522e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.032696 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec712896-76e3-4a2a-a783-4aed7b9522e7 None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.034132 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec712896-76e3-4a2a-a783-4aed7b9522e7 None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:59.035658 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec712896-76e3-4a2a-a783-4aed7b9522e7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:59.037662 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 80/171] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 14:59:58 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:59.038177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec712896-76e3-4a2a-a783-4aed7b9522e7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:59.068819 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60ef09-eff9-48a1-a317-a009531b2260 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:59.106954 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:59.140620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799799f-f1ae-4f95-b82d-6f537567e714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:59.141275 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799799f-f1ae-4f95-b82d-6f537567e714 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.141662 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799799f-f1ae-4f95-b82d-6f537567e714 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.141943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799799f-f1ae-4f95-b82d-6f537567e714 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.175434 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0799799f-f1ae-4f95-b82d-6f537567e714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-0799799f-f1ae-4f95-b82d-6f537567e714 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:59.181460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:59.182344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:59.183272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:59.183272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:59.183272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:59.183272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:59.183272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:59.183272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:59.183272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:59.183272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:59.183272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:59.183272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:59.183272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 14:59:59.183272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 14:59:59.186005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 92/172] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 14:59:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:59.187700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee063c86-734c-42bf-a031-7efc8c5d6d3a None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.187809 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee063c86-734c-42bf-a031-7efc8c5d6d3a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.188059 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee063c86-734c-42bf-a031-7efc8c5d6d3a None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.189456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 93/173] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:59.190241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee063c86-734c-42bf-a031-7efc8c5d6d3a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:59.194565 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec712896-76e3-4a2a-a783-4aed7b9522e7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:59.241520 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbd587-1570-4e9e-a350-4adecb7ee7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:59.243136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbd587-1570-4e9e-a350-4adecb7ee7bd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.243643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbd587-1570-4e9e-a350-4adecb7ee7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.244116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbd587-1570-4e9e-a350-4adecb7ee7bd None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.247408 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbd587-1570-4e9e-a350-4adecb7ee7bd None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:59.251368 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbd587-1570-4e9e-a350-4adecb7ee7bd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:59.256876 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 81/174] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 14:59:59 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:59.260610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbd587-1570-4e9e-a350-4adecb7ee7bd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:59.357659 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee063c86-734c-42bf-a031-7efc8c5d6d3a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:59.380464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:59.391341 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbd587-1570-4e9e-a350-4adecb7ee7bd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:59.397220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f30ac0-c592-4361-ac85-5ee9ae2b6c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:59.397852 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f30ac0-c592-4361-ac85-5ee9ae2b6c53 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.398140 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f30ac0-c592-4361-ac85-5ee9ae2b6c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.398482 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f30ac0-c592-4361-ac85-5ee9ae2b6c53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.399367 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f30ac0-c592-4361-ac85-5ee9ae2b6c53 None admin] RBAC: Adding query filter params (role.id=57221d717a454c7fae4f2df1de9dbe5b, scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80, user.id=9d8a17c4135b4de4bf1d369d28f1c797) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:59.399724 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f30ac0-c592-4361-ac85-5ee9ae2b6c53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:59.401202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f30ac0-c592-4361-ac85-5ee9ae2b6c53 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:59.407422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 94/175] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 14:59:59 2022] GET /identity/v3/role_assignments?user.id=9d8a17c4135b4de4bf1d369d28f1c797&scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80&role.id=57221d717a454c7fae4f2df1de9dbe5b => generated 610 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:59.408220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:59.409104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b40589-c186-4c2d-9bc4-b128e6fa196c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.409549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b40589-c186-4c2d-9bc4-b128e6fa196c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.409869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b40589-c186-4c2d-9bc4-b128e6fa196c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.437050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-82b40589-c186-4c2d-9bc4-b128e6fa196c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:59.438411 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-82b40589-c186-4c2d-9bc4-b128e6fa196c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:59.439823 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225256a4-bac6-4bd3-b4ae-f435c7964a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:59.440692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225256a4-bac6-4bd3-b4ae-f435c7964a6f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.441026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225256a4-bac6-4bd3-b4ae-f435c7964a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.441299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225256a4-bac6-4bd3-b4ae-f435c7964a6f None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.442119 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225256a4-bac6-4bd3-b4ae-f435c7964a6f None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:59.442398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225256a4-bac6-4bd3-b4ae-f435c7964a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:59.443911 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225256a4-bac6-4bd3-b4ae-f435c7964a6f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:59.454331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 82/176] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 14:59:59 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:59.455324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225256a4-bac6-4bd3-b4ae-f435c7964a6f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:59.524797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 95/177] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:59.532292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59a1e-2bb5-4fd2-86f6-d2a7eeb1da25 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.533036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59a1e-2bb5-4fd2-86f6-d2a7eeb1da25 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.533036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59a1e-2bb5-4fd2-86f6-d2a7eeb1da25 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.534481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 96/178] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:59 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:59.553331 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17a322-ba52-4913-b59d-019fc4e85131 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.553617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17a322-ba52-4913-b59d-019fc4e85131 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.553893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17a322-ba52-4913-b59d-019fc4e85131 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.555981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 97/179] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:59 2022] GET /identity => generated 271 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:59.563867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17a322-ba52-4913-b59d-019fc4e85131 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:59.604485 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225256a4-bac6-4bd3-b4ae-f435c7964a6f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:59.651787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4ddfe-d536-492e-92c2-caa69c23b8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:59.652601 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4ddfe-d536-492e-92c2-caa69c23b8af None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.652601 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4ddfe-d536-492e-92c2-caa69c23b8af None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.652879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4ddfe-d536-492e-92c2-caa69c23b8af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.653667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4ddfe-d536-492e-92c2-caa69c23b8af None admin] RBAC: Adding query filter params (role.id=d1f6973ce11a4644b649b38f8b2df173, scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80, user.id=9d8a17c4135b4de4bf1d369d28f1c797) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 14:59:59.653978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4ddfe-d536-492e-92c2-caa69c23b8af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:59.655501 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4ddfe-d536-492e-92c2-caa69c23b8af None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:59.662013 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 83/180] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 14:59:59 2022] GET /identity/v3/role_assignments?user.id=9d8a17c4135b4de4bf1d369d28f1c797&scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80&role.id=d1f6973ce11a4644b649b38f8b2df173 => generated 610 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:59.664233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a40112-9603-4864-8738-18fc19633651 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.664233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a40112-9603-4864-8738-18fc19633651 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.664233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a40112-9603-4864-8738-18fc19633651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.684645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a0a40112-9603-4864-8738-18fc19633651 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:59.685888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a0a40112-9603-4864-8738-18fc19633651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:59.719808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17a322-ba52-4913-b59d-019fc4e85131 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:59.736818 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:59.738098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 14:59:59.763382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4048d3-07b7-48d8-9aa2-119d363fc2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:59.764049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4048d3-07b7-48d8-9aa2-119d363fc2b3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.764327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4048d3-07b7-48d8-9aa2-119d363fc2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.764601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4048d3-07b7-48d8-9aa2-119d363fc2b3 None admin] PATH_INFO: `/v3/users/9d8a17c4135b4de4bf1d369d28f1c797` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.765827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4048d3-07b7-48d8-9aa2-119d363fc2b3 None admin] RBAC: Authorizing `identity:get_user(user_id=9d8a17c4135b4de4bf1d369d28f1c797)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 14:59:59.767482 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4048d3-07b7-48d8-9aa2-119d363fc2b3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 14:59:59.769205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 98/181] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 14:59:59 2022] GET /identity/v3/users/9d8a17c4135b4de4bf1d369d28f1c797 => generated 276 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 14:59:59.770456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8aa3c-237e-4536-b526-4cf86aee70f8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.770771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8aa3c-237e-4536-b526-4cf86aee70f8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.771044 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8aa3c-237e-4536-b526-4cf86aee70f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.783901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 84/182] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:59.785171 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a40112-9603-4864-8738-18fc19633651 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:59.790366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a1a8aa3c-237e-4536-b526-4cf86aee70f8 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 14:59:59.795291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a1a8aa3c-237e-4536-b526-4cf86aee70f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 14:59:59.872886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 99/183] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 14:59:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 14:59:59.874016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e36d4-1183-4b21-a006-2554514b7592 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.874305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e36d4-1183-4b21-a006-2554514b7592 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.874585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e36d4-1183-4b21-a006-2554514b7592 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.876077 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 100/184] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 14:59:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 14:59:59.876596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e36d4-1183-4b21-a006-2554514b7592 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:59.912985 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a40112-9603-4864-8738-18fc19633651 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 14:59:59.951734 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3eb399-bae0-46a6-a89d-e89cd4ffdc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 14:59:59.952383 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3eb399-bae0-46a6-a89d-e89cd4ffdc2c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 14:59:59.952664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3eb399-bae0-46a6-a89d-e89cd4ffdc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 14:59:59.952936 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3eb399-bae0-46a6-a89d-e89cd4ffdc2c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 14:59:59.963455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3eb399-bae0-46a6-a89d-e89cd4ffdc2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-fd3eb399-bae0-46a6-a89d-e89cd4ffdc2c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 14:59:59.968381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 14:59:59.969299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 14:59:59.970382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 14:59:59.970382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 14:59:59.970382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 14:59:59.970382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 14:59:59.970382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 14:59:59.970382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 14:59:59.970382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 14:59:59.970382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 14:59:59.970382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 14:59:59.970382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 14:59:59.970382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 14:59:59.970382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 14:59:59.970382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 85/185] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 14:59:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 14:59:59.971249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3eb399-bae0-46a6-a89d-e89cd4ffdc2c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 14:59:59.997730 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e36d4-1183-4b21-a006-2554514b7592 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:00.001784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:00.011511 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:00.033192 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b708f-ed80-4516-b145-b362a5bb7d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:00.033813 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b708f-ed80-4516-b145-b362a5bb7d41 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:00.034096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b708f-ed80-4516-b145-b362a5bb7d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:00.034371 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b708f-ed80-4516-b145-b362a5bb7d41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:00.073920 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3eb399-bae0-46a6-a89d-e89cd4ffdc2c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:00.106380 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b708f-ed80-4516-b145-b362a5bb7d41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:00.107914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b708f-ed80-4516-b145-b362a5bb7d41 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:00.118881 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c407409-0fec-4f4d-89c2-ba44d3188349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:00.119533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407409-0fec-4f4d-89c2-ba44d3188349 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:00.119812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407409-0fec-4f4d-89c2-ba44d3188349 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:00.120084 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c407409-0fec-4f4d-89c2-ba44d3188349 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:00.130592 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c407409-0fec-4f4d-89c2-ba44d3188349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7c407409-0fec-4f4d-89c2-ba44d3188349 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:00.135591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:00.143270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:00.144502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:00.144502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:00.144502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:00.144502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:00.144502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:00.144502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:00.144502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:00.144502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:00.144502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:00.144502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:00.144502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:00.144502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:00.144502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 86/186] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 14:59:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:00.144502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd7859-7af6-425e-919c-4c1240e3df09 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:00.145545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd7859-7af6-425e-919c-4c1240e3df09 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:00.145545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd7859-7af6-425e-919c-4c1240e3df09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:00.156624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5ebd7859-7af6-425e-919c-4c1240e3df09 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:00.157983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5ebd7859-7af6-425e-919c-4c1240e3df09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:00.226455 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 101/187] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 14:59:59 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:00.228021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b708f-ed80-4516-b145-b362a5bb7d41 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:00.240306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 87/188] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:00.241160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd7859-7af6-425e-919c-4c1240e3df09 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:00.248562 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:00.268853 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:00.340880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b708f-ed80-4516-b145-b362a5bb7d41 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:00.351485 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd7859-7af6-425e-919c-4c1240e3df09 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:00.375150 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1201f-6326-475d-aee5-494d9e56ec0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:00.375804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1201f-6326-475d-aee5-494d9e56ec0a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:00.376067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1201f-6326-475d-aee5-494d9e56ec0a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:00.376343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1201f-6326-475d-aee5-494d9e56ec0a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:00.385471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd92380-7cdd-4435-9b84-3809fc92f824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:00.386085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd92380-7cdd-4435-9b84-3809fc92f824 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:00.386363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd92380-7cdd-4435-9b84-3809fc92f824 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:00.386456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1201f-6326-475d-aee5-494d9e56ec0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:00.386638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd92380-7cdd-4435-9b84-3809fc92f824 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:00.387392 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd92380-7cdd-4435-9b84-3809fc92f824 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:00.387697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd92380-7cdd-4435-9b84-3809fc92f824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:00.389080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd92380-7cdd-4435-9b84-3809fc92f824 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-b4c1201f-6326-475d-aee5-494d9e56ec0a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:00.391531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:00.392609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:00.393588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:00.393588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:00.393588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:00.393588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:00.393588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:00.393588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:00.393588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:00.393588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:00.393588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:00.393588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:00.393588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:00:00.393588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:00.393588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 102/189] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:00.394499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1201f-6326-475d-aee5-494d9e56ec0a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:00.402527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 88/190] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:00 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:00.403435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd92380-7cdd-4435-9b84-3809fc92f824 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:00.465562 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:00.489175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:00.498557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1201f-6326-475d-aee5-494d9e56ec0a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:00.509191 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd92380-7cdd-4435-9b84-3809fc92f824 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:00.533877 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bedd3-8397-4496-853d-ddc32220ba4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:00.534495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bedd3-8397-4496-853d-ddc32220ba4d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:00.534805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bedd3-8397-4496-853d-ddc32220ba4d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:00.535094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bedd3-8397-4496-853d-ddc32220ba4d None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:00.535886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8bedd3-8397-4496-853d-ddc32220ba4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:00.536090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8bedd3-8397-4496-853d-ddc32220ba4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:00.537479 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8bedd3-8397-4496-853d-ddc32220ba4d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:00.547053 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63798c04-3206-41bb-b370-fc0e81c758ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:00.547886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63798c04-3206-41bb-b370-fc0e81c758ef None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:00.548010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 103/191] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:00 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:00.548218 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63798c04-3206-41bb-b370-fc0e81c758ef None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:00.548424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63798c04-3206-41bb-b370-fc0e81c758ef None admin] PATH_INFO: `/v3/projects/0ab4610904d94edd8eb3a1f420c13314` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:00.549623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63798c04-3206-41bb-b370-fc0e81c758ef None admin] RBAC: Authorizing `identity:get_project(project_id=0ab4610904d94edd8eb3a1f420c13314)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:00.551121 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63798c04-3206-41bb-b370-fc0e81c758ef None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:00.552949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 89/192] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:00 2022] GET /identity/v3/projects/0ab4610904d94edd8eb3a1f420c13314 => generated 313 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:00.553359 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bedd3-8397-4496-853d-ddc32220ba4d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:00.553550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63798c04-3206-41bb-b370-fc0e81c758ef None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:00.659362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bedd3-8397-4496-853d-ddc32220ba4d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:00.663590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63798c04-3206-41bb-b370-fc0e81c758ef None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:00.694031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca4be4-1431-422f-8824-4edfa62c3e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:00.694693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca4be4-1431-422f-8824-4edfa62c3e2c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:00.694985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca4be4-1431-422f-8824-4edfa62c3e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:00.695264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca4be4-1431-422f-8824-4edfa62c3e2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:00.696934 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca4be4-1431-422f-8824-4edfa62c3e2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:00.698409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca4be4-1431-422f-8824-4edfa62c3e2c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:00.700060 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 104/193] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:00 2022] GET /identity/v3/domains/default => generated 200 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:00.700917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca4be4-1431-422f-8824-4edfa62c3e2c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:00.703658 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:00.707238 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca00cb-31e3-41ed-8831-a4d15e343591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:00.707869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca00cb-31e3-41ed-8831-a4d15e343591 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:00.708150 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca00cb-31e3-41ed-8831-a4d15e343591 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:00.708420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca00cb-31e3-41ed-8831-a4d15e343591 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:00.709144 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca00cb-31e3-41ed-8831-a4d15e343591 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:00.709442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca00cb-31e3-41ed-8831-a4d15e343591 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:00.710873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca00cb-31e3-41ed-8831-a4d15e343591 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:00.721094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 90/194] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:00 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:00.722027 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca00cb-31e3-41ed-8831-a4d15e343591 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:00.738086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:00.809940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca4be4-1431-422f-8824-4edfa62c3e2c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:00.828624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca00cb-31e3-41ed-8831-a4d15e343591 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:00.848938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a6aa9-dd5c-4f31-ac02-8c9500804275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:00.849918 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a6aa9-dd5c-4f31-ac02-8c9500804275 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:00.850288 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a6aa9-dd5c-4f31-ac02-8c9500804275 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:00.850575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a6aa9-dd5c-4f31-ac02-8c9500804275 None admin] PATH_INFO: `/v3/users/b519d0db93834e6491f62b7d72ce0dc6` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:00.863848 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551e1fd-a2f8-44ff-add5-4b8c4ba84673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:00.864498 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e1fd-a2f8-44ff-add5-4b8c4ba84673 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:00.864775 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e1fd-a2f8-44ff-add5-4b8c4ba84673 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:00.865050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e1fd-a2f8-44ff-add5-4b8c4ba84673 None admin] PATH_INFO: `/v3/users/b519d0db93834e6491f62b7d72ce0dc6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:00.865913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a6aa9-dd5c-4f31-ac02-8c9500804275 None admin] RBAC: Authorizing `identity:get_user(user_id=b519d0db93834e6491f62b7d72ce0dc6)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:00.866239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551e1fd-a2f8-44ff-add5-4b8c4ba84673 None admin] RBAC: Authorizing `identity:get_user(user_id=b519d0db93834e6491f62b7d72ce0dc6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:00.867720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551e1fd-a2f8-44ff-add5-4b8c4ba84673 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:00.867843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a6aa9-dd5c-4f31-ac02-8c9500804275 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:00.869392 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 91/195] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:00 2022] GET /identity/v3/users/b519d0db93834e6491f62b7d72ce0dc6 => generated 284 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:00.869809 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 105/196] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:00 2022] GET /identity/v3/users/b519d0db93834e6491f62b7d72ce0dc6 => generated 284 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:00.870252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551e1fd-a2f8-44ff-add5-4b8c4ba84673 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:00.870424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a6aa9-dd5c-4f31-ac02-8c9500804275 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:00.938392 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:00.958476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:00.980865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551e1fd-a2f8-44ff-add5-4b8c4ba84673 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:00.983712 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a6aa9-dd5c-4f31-ac02-8c9500804275 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:01.019030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10554f-f1da-44e5-932c-fc1baebbca97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:01.019647 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10554f-f1da-44e5-932c-fc1baebbca97 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:01.019889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd6110-5cc8-4b99-abe9-f99ff3e2f9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:01.019998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10554f-f1da-44e5-932c-fc1baebbca97 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:01.020179 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10554f-f1da-44e5-932c-fc1baebbca97 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:01.020591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd6110-5cc8-4b99-abe9-f99ff3e2f9eb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:01.020868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10554f-f1da-44e5-932c-fc1baebbca97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:01.020957 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd6110-5cc8-4b99-abe9-f99ff3e2f9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:01.021208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd6110-5cc8-4b99-abe9-f99ff3e2f9eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:01.022199 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd6110-5cc8-4b99-abe9-f99ff3e2f9eb None admin] RBAC: Adding query filter params (role.id=d1f6973ce11a4644b649b38f8b2df173, scope.project.id=0ab4610904d94edd8eb3a1f420c13314, user.id=9d8a17c4135b4de4bf1d369d28f1c797) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:01.022305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10554f-f1da-44e5-932c-fc1baebbca97 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:01.022508 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd6110-5cc8-4b99-abe9-f99ff3e2f9eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:01.024480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cd6110-5cc8-4b99-abe9-f99ff3e2f9eb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:01.031643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 92/197] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:00 2022] GET /identity/v3/role_assignments?user.id=9d8a17c4135b4de4bf1d369d28f1c797&scope.project.id=0ab4610904d94edd8eb3a1f420c13314&role.id=d1f6973ce11a4644b649b38f8b2df173 => generated 610 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:01.032536 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd6110-5cc8-4b99-abe9-f99ff3e2f9eb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:01.050880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 106/198] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:00:00 2022] POST /identity/v3/users => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:00:01.051949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10554f-f1da-44e5-932c-fc1baebbca97 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:01.184249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cd6110-5cc8-4b99-abe9-f99ff3e2f9eb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:01.189141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10554f-f1da-44e5-932c-fc1baebbca97 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:01.217300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:01.229417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aad7e-d19d-4af3-adc5-115c1a7e7120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:01.230453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aad7e-d19d-4af3-adc5-115c1a7e7120 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:01.231007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aad7e-d19d-4af3-adc5-115c1a7e7120 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:01.231613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aad7e-d19d-4af3-adc5-115c1a7e7120 None admin] PATH_INFO: `/v3/users/64cfd0d9528a405688a7d6185ba97242` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:01.233305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6aad7e-d19d-4af3-adc5-115c1a7e7120 None admin] RBAC: Authorizing `identity:get_user(user_id=64cfd0d9528a405688a7d6185ba97242)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:01.235198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6aad7e-d19d-4af3-adc5-115c1a7e7120 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:01.236577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd7319-1ccc-440d-8913-8214740a47f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:01.237380 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd7319-1ccc-440d-8913-8214740a47f6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:01.237540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 93/199] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:01 2022] GET /identity/v3/users/64cfd0d9528a405688a7d6185ba97242 => generated 290 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:01.238079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd7319-1ccc-440d-8913-8214740a47f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:01.238512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aad7e-d19d-4af3-adc5-115c1a7e7120 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:01.238860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd7319-1ccc-440d-8913-8214740a47f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:01.258000 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:01.341385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd7319-1ccc-440d-8913-8214740a47f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:01.342866 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd7319-1ccc-440d-8913-8214740a47f6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:01.397299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aad7e-d19d-4af3-adc5-115c1a7e7120 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:01.451946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e96341-d923-45b2-9c91-09142a865ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:01.452512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e96341-d923-45b2-9c91-09142a865ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:01.452799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e96341-d923-45b2-9c91-09142a865ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:01.453080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e96341-d923-45b2-9c91-09142a865ba2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:01.502817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 107/200] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:01 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:01.506013 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd7319-1ccc-440d-8913-8214740a47f6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:01.532753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:01.565955 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e96341-d923-45b2-9c91-09142a865ba2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:01.568381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e96341-d923-45b2-9c91-09142a865ba2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:01.590339 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:01.671125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd7319-1ccc-440d-8913-8214740a47f6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:01.710561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 94/201] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:01 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:01.715152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa488fd-29f1-4432-88bf-b9dcda3715e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:01.716065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa488fd-29f1-4432-88bf-b9dcda3715e6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:01.716663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa488fd-29f1-4432-88bf-b9dcda3715e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:01.717151 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa488fd-29f1-4432-88bf-b9dcda3715e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:01.717546 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e96341-d923-45b2-9c91-09142a865ba2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:01.797337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa488fd-29f1-4432-88bf-b9dcda3715e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:01.799224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa488fd-29f1-4432-88bf-b9dcda3715e6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:01.819432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:01.840483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e96341-d923-45b2-9c91-09142a865ba2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:01.854876 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:01.878454 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccead08-2815-40ab-9dbc-2bf8a1a776a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:01.879283 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccead08-2815-40ab-9dbc-2bf8a1a776a7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:01.879563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccead08-2815-40ab-9dbc-2bf8a1a776a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:01.879842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccead08-2815-40ab-9dbc-2bf8a1a776a7 None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:01.881302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccead08-2815-40ab-9dbc-2bf8a1a776a7 None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:01.883132 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccead08-2815-40ab-9dbc-2bf8a1a776a7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:01.885625 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 95/202] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:01 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:01.890494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccead08-2815-40ab-9dbc-2bf8a1a776a7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:01.931779 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 108/203] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:01 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:01.933222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa488fd-29f1-4432-88bf-b9dcda3715e6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:02.028772 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccead08-2815-40ab-9dbc-2bf8a1a776a7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:02.049777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa488fd-29f1-4432-88bf-b9dcda3715e6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:02.068996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:02.072641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be809d0-9e21-49c4-b54f-99927303a029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:02.073396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be809d0-9e21-49c4-b54f-99927303a029 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:02.073679 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be809d0-9e21-49c4-b54f-99927303a029 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:02.073948 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be809d0-9e21-49c4-b54f-99927303a029 None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:02.076033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be809d0-9e21-49c4-b54f-99927303a029 None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:02.077650 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be809d0-9e21-49c4-b54f-99927303a029 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:02.079779 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 96/204] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:01 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:02.085562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be809d0-9e21-49c4-b54f-99927303a029 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:02.090852 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f9e50-01b4-4a0d-9694-00d2f7b5e169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:02.091587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f9e50-01b4-4a0d-9694-00d2f7b5e169 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:02.091934 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f9e50-01b4-4a0d-9694-00d2f7b5e169 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:02.092276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f9e50-01b4-4a0d-9694-00d2f7b5e169 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:02.093141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f9e50-01b4-4a0d-9694-00d2f7b5e169 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f, user.id=b519d0db93834e6491f62b7d72ce0dc6) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:02.093519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f9e50-01b4-4a0d-9694-00d2f7b5e169 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:02.095184 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f9e50-01b4-4a0d-9694-00d2f7b5e169 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:02.102141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 109/205] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:01 2022] GET /identity/v3/role_assignments?user.id=b519d0db93834e6491f62b7d72ce0dc6&scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:02.103101 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f9e50-01b4-4a0d-9694-00d2f7b5e169 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:02.149825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:02.213504 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be809d0-9e21-49c4-b54f-99927303a029 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:02.223805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f9e50-01b4-4a0d-9694-00d2f7b5e169 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:02.254523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8f8de-1056-4ac4-9da4-2fdc1f509f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:02.255230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8f8de-1056-4ac4-9da4-2fdc1f509f69 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:02.255519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8f8de-1056-4ac4-9da4-2fdc1f509f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:02.255797 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8f8de-1056-4ac4-9da4-2fdc1f509f69 None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:02.257336 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8f8de-1056-4ac4-9da4-2fdc1f509f69 None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:02.259657 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8f8de-1056-4ac4-9da4-2fdc1f509f69 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:02.261484 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 97/206] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:02 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:02.265070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ac587-92fe-468b-abe1-ca52914e9cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:02.265966 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ac587-92fe-468b-abe1-ca52914e9cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:02.266816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ac587-92fe-468b-abe1-ca52914e9cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:02.267311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ac587-92fe-468b-abe1-ca52914e9cd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:02.267641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8f8de-1056-4ac4-9da4-2fdc1f509f69 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:02.268569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ac587-92fe-468b-abe1-ca52914e9cd7 None admin] RBAC: Adding query filter params (role.id=57221d717a454c7fae4f2df1de9dbe5b, scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f, user.id=b519d0db93834e6491f62b7d72ce0dc6) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:02.269093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ac587-92fe-468b-abe1-ca52914e9cd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:02.270928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ac587-92fe-468b-abe1-ca52914e9cd7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:02.278271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 110/207] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:02 2022] GET /identity/v3/role_assignments?user.id=b519d0db93834e6491f62b7d72ce0dc6&scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f&role.id=57221d717a454c7fae4f2df1de9dbe5b => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:02.414916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8f8de-1056-4ac4-9da4-2fdc1f509f69 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:02.439932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:02.458433 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910f85f-a093-4458-a5d3-9f1ab7eda74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:02.459234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910f85f-a093-4458-a5d3-9f1ab7eda74b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:02.459576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910f85f-a093-4458-a5d3-9f1ab7eda74b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:02.459860 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910f85f-a093-4458-a5d3-9f1ab7eda74b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:02.460759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910f85f-a093-4458-a5d3-9f1ab7eda74b None admin] RBAC: Adding query filter params (role.id=471fd358281343cba97902b70b416f32, scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80, user.id=64cfd0d9528a405688a7d6185ba97242) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:02.461686 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910f85f-a093-4458-a5d3-9f1ab7eda74b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:02.463662 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5910f85f-a093-4458-a5d3-9f1ab7eda74b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:02.471036 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 98/208] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:02 2022] GET /identity/v3/role_assignments?user.id=64cfd0d9528a405688a7d6185ba97242&scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80&role.id=471fd358281343cba97902b70b416f32 => generated 610 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:03.645249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dac2e5-ce47-4a0e-8478-bdb141191ea8 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:03.645776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dac2e5-ce47-4a0e-8478-bdb141191ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:03.646031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dac2e5-ce47-4a0e-8478-bdb141191ea8 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:03.647774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 111/209] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:03.654350 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdeb78-3476-40d6-90ce-aeb43c6a851a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:03.655562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdeb78-3476-40d6-90ce-aeb43c6a851a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:03.656099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdeb78-3476-40d6-90ce-aeb43c6a851a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:03.676951 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a7bdeb78-3476-40d6-90ce-aeb43c6a851a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:03.678469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a7bdeb78-3476-40d6-90ce-aeb43c6a851a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:03.735069 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f0f12-46a7-4436-a751-09ebd9465c50 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:03.735361 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f0f12-46a7-4436-a751-09ebd9465c50 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:03.736071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766f0f12-46a7-4436-a751-09ebd9465c50 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:03.737274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 112/210] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:03.744997 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b21c05-d46f-4a22-869f-ec442461af20 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:03.745273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b21c05-d46f-4a22-869f-ec442461af20 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:03.745554 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b21c05-d46f-4a22-869f-ec442461af20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:03.763901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c4b21c05-d46f-4a22-869f-ec442461af20 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:03.765111 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c4b21c05-d46f-4a22-869f-ec442461af20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:03.770743 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 99/211] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:03.793415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bdeb78-3476-40d6-90ce-aeb43c6a851a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:03.816426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:03.850962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 113/212] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:03.874956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b21c05-d46f-4a22-869f-ec442461af20 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:03.917041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bdeb78-3476-40d6-90ce-aeb43c6a851a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:03.928919 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:03.955767 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38430253-32ed-47b7-bfd1-227d28126eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:03.956421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38430253-32ed-47b7-bfd1-227d28126eca None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:03.956701 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38430253-32ed-47b7-bfd1-227d28126eca None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:03.956973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38430253-32ed-47b7-bfd1-227d28126eca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:03.975862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38430253-32ed-47b7-bfd1-227d28126eca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-38430253-32ed-47b7-bfd1-227d28126eca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:03.985862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:03.986859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:03.987680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:03.987680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:03.987680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:03.987680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:03.987680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:03.987680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:03.987680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:03.987680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:03.987680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:03.987680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:03.987680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:03.987680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:03.987680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 100/213] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:03.990913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b21c05-d46f-4a22-869f-ec442461af20 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:03.993803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38430253-32ed-47b7-bfd1-227d28126eca None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:04.049271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b706d-cfea-4dfc-989b-cb481b9cd0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:04.051056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b706d-cfea-4dfc-989b-cb481b9cd0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:04.051896 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b706d-cfea-4dfc-989b-cb481b9cd0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:04.052587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b706d-cfea-4dfc-989b-cb481b9cd0e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:04.055243 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b706d-cfea-4dfc-989b-cb481b9cd0e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:04.058186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b706d-cfea-4dfc-989b-cb481b9cd0e7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:04.067122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 114/214] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:03 2022] GET /identity/v3/domains/default => generated 200 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:04.071679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b706d-cfea-4dfc-989b-cb481b9cd0e7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:04.122392 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38430253-32ed-47b7-bfd1-227d28126eca None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:04.153756 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:04.171028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdf41f-8a57-4e8a-a8bd-c65d518925cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:04.171669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdf41f-8a57-4e8a-a8bd-c65d518925cc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:04.171966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdf41f-8a57-4e8a-a8bd-c65d518925cc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:04.172285 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdf41f-8a57-4e8a-a8bd-c65d518925cc None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:04.173024 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdf41f-8a57-4e8a-a8bd-c65d518925cc None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:04.173327 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdf41f-8a57-4e8a-a8bd-c65d518925cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:04.174756 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdf41f-8a57-4e8a-a8bd-c65d518925cc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:04.186835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 101/215] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:03 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:04.194328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdf41f-8a57-4e8a-a8bd-c65d518925cc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:04.239111 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:04.239860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b706d-cfea-4dfc-989b-cb481b9cd0e7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:04.279067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1d8bb-1709-46bf-a300-8eda75273d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:04.280090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1d8bb-1709-46bf-a300-8eda75273d26 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:04.280573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1d8bb-1709-46bf-a300-8eda75273d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:04.281048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1d8bb-1709-46bf-a300-8eda75273d26 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:04.282037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1d8bb-1709-46bf-a300-8eda75273d26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:04.283865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c1d8bb-1709-46bf-a300-8eda75273d26 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:04.301634 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdf41f-8a57-4e8a-a8bd-c65d518925cc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:04.315616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 115/216] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:00:04 2022] POST /identity/v3/users => generated 298 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:00:04.345999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020539e4-30f6-4578-aa3a-bedae6646ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:04.346692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020539e4-30f6-4578-aa3a-bedae6646ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:04.347025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020539e4-30f6-4578-aa3a-bedae6646ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:04.347340 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020539e4-30f6-4578-aa3a-bedae6646ed0 None admin] PATH_INFO: `/v3/users/d30378f810a1497f94191b23bd742912` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:04.365736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020539e4-30f6-4578-aa3a-bedae6646ed0 None admin] RBAC: Authorizing `identity:get_user(user_id=d30378f810a1497f94191b23bd742912)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:04.367456 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020539e4-30f6-4578-aa3a-bedae6646ed0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:04.369674 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 102/217] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:04 2022] GET /identity/v3/users/d30378f810a1497f94191b23bd742912 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:04.376120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1d8bb-1709-46bf-a300-8eda75273d26 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:04.478803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d67dfe-a5f9-4a69-8c39-84bf0a64ee83 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:04.479237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d67dfe-a5f9-4a69-8c39-84bf0a64ee83 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:04.479401 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d67dfe-a5f9-4a69-8c39-84bf0a64ee83 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:04.481474 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 103/218] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:04 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:04.487891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e3dc-8b28-475f-a880-e75d8cfe5e33 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:04.488190 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e3dc-8b28-475f-a880-e75d8cfe5e33 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:04.488443 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475e3dc-8b28-475f-a880-e75d8cfe5e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:04.513334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:04.514257 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6475e3dc-8b28-475f-a880-e75d8cfe5e33 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:04.515198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6475e3dc-8b28-475f-a880-e75d8cfe5e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:04.518939 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1d8bb-1709-46bf-a300-8eda75273d26 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:04.568656 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112debc3-a116-4fcd-9189-f6b62e63d536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:04.568843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112debc3-a116-4fcd-9189-f6b62e63d536 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:04.569031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112debc3-a116-4fcd-9189-f6b62e63d536 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:04.569349 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112debc3-a116-4fcd-9189-f6b62e63d536 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:04.610009 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 104/219] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:04 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:00:04.633682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475e3dc-8b28-475f-a880-e75d8cfe5e33 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:04.665511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:04.666129 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112debc3-a116-4fcd-9189-f6b62e63d536 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:04.668878 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112debc3-a116-4fcd-9189-f6b62e63d536 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:04.769129 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475e3dc-8b28-475f-a880-e75d8cfe5e33 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:04.784330 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:04.791043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 116/220] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:04 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:04.796742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112debc3-a116-4fcd-9189-f6b62e63d536 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:04.821566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ae481-6d3e-4b09-8965-486266624ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:04.822686 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ae481-6d3e-4b09-8965-486266624ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:04.823193 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ae481-6d3e-4b09-8965-486266624ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:04.824344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ae481-6d3e-4b09-8965-486266624ca0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:04.841045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ae481-6d3e-4b09-8965-486266624ca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-235ae481-6d3e-4b09-8965-486266624ca0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:04.846409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:04.847420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:04.849008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:04.849008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:04.849008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:04.849008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:04.849008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:04.849008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:04.849008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:04.849008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:04.849008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:04.849008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:04.849008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:04.849008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:04.849008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 105/221] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:00:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:04.849659 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0ad95-3afc-484f-8132-24874a2da18a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:04.849659 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0ad95-3afc-484f-8132-24874a2da18a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:04.849787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d0ad95-3afc-484f-8132-24874a2da18a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:04.851450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 106/222] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:04.855804 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0ad95-3afc-484f-8132-24874a2da18a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:04.929820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112debc3-a116-4fcd-9189-f6b62e63d536 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:04.958540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:04.969562 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987b5b6-4224-4210-9dd8-017bfa3dd2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:04.970314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987b5b6-4224-4210-9dd8-017bfa3dd2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:04.970689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987b5b6-4224-4210-9dd8-017bfa3dd2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:04.971033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987b5b6-4224-4210-9dd8-017bfa3dd2b0 None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:04.972598 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b987b5b6-4224-4210-9dd8-017bfa3dd2b0 None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:04.973959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b987b5b6-4224-4210-9dd8-017bfa3dd2b0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:04.975984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 117/223] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:04 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:04.982333 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14533136-a2b2-410d-b653-abe2203fb7a8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:04.982726 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14533136-a2b2-410d-b653-abe2203fb7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:04.983068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14533136-a2b2-410d-b653-abe2203fb7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:04.989122 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d0ad95-3afc-484f-8132-24874a2da18a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:05.001639 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-14533136-a2b2-410d-b653-abe2203fb7a8 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:05.003189 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-14533136-a2b2-410d-b653-abe2203fb7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:05.035886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4f792-0d2b-48b8-952e-2ae875b62fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:05.036970 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4f792-0d2b-48b8-952e-2ae875b62fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:05.037510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4f792-0d2b-48b8-952e-2ae875b62fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:05.037953 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4f792-0d2b-48b8-952e-2ae875b62fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:05.038995 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4f792-0d2b-48b8-952e-2ae875b62fe9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:05.039996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4f792-0d2b-48b8-952e-2ae875b62fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:05.042983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4f792-0d2b-48b8-952e-2ae875b62fe9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:05.055615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 107/224] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:04 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:05.057009 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4f792-0d2b-48b8-952e-2ae875b62fe9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:05.071360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:05.083348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 118/225] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:04 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:05.085131 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14533136-a2b2-410d-b653-abe2203fb7a8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:05.193246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4f792-0d2b-48b8-952e-2ae875b62fe9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:05.223299 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14533136-a2b2-410d-b653-abe2203fb7a8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:05.240528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae86f7-5b97-48df-b28d-aaf2215be5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:05.241159 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae86f7-5b97-48df-b28d-aaf2215be5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:05.241444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae86f7-5b97-48df-b28d-aaf2215be5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:05.241722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae86f7-5b97-48df-b28d-aaf2215be5f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:05.242548 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae86f7-5b97-48df-b28d-aaf2215be5f0 None admin] RBAC: Adding query filter params (role.id=d1f6973ce11a4644b649b38f8b2df173, scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f, user.id=d30378f810a1497f94191b23bd742912) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:05.242898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae86f7-5b97-48df-b28d-aaf2215be5f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:05.244354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae86f7-5b97-48df-b28d-aaf2215be5f0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:05.250389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 108/226] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:05 2022] GET /identity/v3/role_assignments?user.id=d30378f810a1497f94191b23bd742912&scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f&role.id=d1f6973ce11a4644b649b38f8b2df173 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:05.259680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae86f7-5b97-48df-b28d-aaf2215be5f0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:05.268838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:05.270163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f18875-e1dc-4f4e-ac31-8cca5afffaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:05.271254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f18875-e1dc-4f4e-ac31-8cca5afffaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:05.271747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f18875-e1dc-4f4e-ac31-8cca5afffaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:05.272215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f18875-e1dc-4f4e-ac31-8cca5afffaf0 None admin] PATH_INFO: `/v3/users/b519d0db93834e6491f62b7d72ce0dc6` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:05.274247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f18875-e1dc-4f4e-ac31-8cca5afffaf0 None admin] RBAC: Authorizing `identity:get_user(user_id=b519d0db93834e6491f62b7d72ce0dc6)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:05.276066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f18875-e1dc-4f4e-ac31-8cca5afffaf0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:05.278092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 119/227] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:05 2022] GET /identity/v3/users/b519d0db93834e6491f62b7d72ce0dc6 => generated 284 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:05.287971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f18875-e1dc-4f4e-ac31-8cca5afffaf0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:05.372644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:05.437929 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f18875-e1dc-4f4e-ac31-8cca5afffaf0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:05.506201 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae86f7-5b97-48df-b28d-aaf2215be5f0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:05.518005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb8750-1384-40c6-a2bf-fecfcf3f0e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:05.519056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb8750-1384-40c6-a2bf-fecfcf3f0e2e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:05.519439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb8750-1384-40c6-a2bf-fecfcf3f0e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:05.519806 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eb8750-1384-40c6-a2bf-fecfcf3f0e2e None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:05.521632 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb8750-1384-40c6-a2bf-fecfcf3f0e2e None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:05.524099 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eb8750-1384-40c6-a2bf-fecfcf3f0e2e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:05.526583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 120/228] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:05 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:05.549097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb8750-1384-40c6-a2bf-fecfcf3f0e2e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:05.554018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6783c2-9e31-418d-afe6-fecf6cce79cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:05.555258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6783c2-9e31-418d-afe6-fecf6cce79cb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:05.555543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6783c2-9e31-418d-afe6-fecf6cce79cb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:05.555820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6783c2-9e31-418d-afe6-fecf6cce79cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:05.566998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6783c2-9e31-418d-afe6-fecf6cce79cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7b6783c2-9e31-418d-afe6-fecf6cce79cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:05.573233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:05.574109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:05.574935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:05.574935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:05.574935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:05.574935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:05.574935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:05.574935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:05.574935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:05.574935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:05.574935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:05.574935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:05.574935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:05.574935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:05.574935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 109/229] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:00:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:05.584058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6783c2-9e31-418d-afe6-fecf6cce79cb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:05.727057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:05.757234 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eb8750-1384-40c6-a2bf-fecfcf3f0e2e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:05.780998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6783c2-9e31-418d-afe6-fecf6cce79cb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:05.829542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:05.837724 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf11d7-fd00-4eef-b31c-776d7184821b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:05.839596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf11d7-fd00-4eef-b31c-776d7184821b None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:05.840164 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf11d7-fd00-4eef-b31c-776d7184821b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:05.841194 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf11d7-fd00-4eef-b31c-776d7184821b None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f/users/b519d0db93834e6491f62b7d72ce0dc6/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:05.844824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf11d7-fd00-4eef-b31c-776d7184821b None admin] RBAC: Authorizing `identity:create_grant(project_id=ebb7ad89c9734ecbbd41c0c2de27069f, user_id=b519d0db93834e6491f62b7d72ce0dc6, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:05.846960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf11d7-fd00-4eef-b31c-776d7184821b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:05.853576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e757d5-4330-4a69-9b01-d4fdbc570539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:05.854279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e757d5-4330-4a69-9b01-d4fdbc570539 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:05.854576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e757d5-4330-4a69-9b01-d4fdbc570539 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:05.854912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e757d5-4330-4a69-9b01-d4fdbc570539 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:05.855660 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e757d5-4330-4a69-9b01-d4fdbc570539 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:05.856020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e757d5-4330-4a69-9b01-d4fdbc570539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:05.857606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e757d5-4330-4a69-9b01-d4fdbc570539 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:05.864008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 121/230] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:00:05 2022] PUT /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f/users/b519d0db93834e6491f62b7d72ce0dc6/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:05.877271 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 110/231] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:05 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:05.883264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf11d7-fd00-4eef-b31c-776d7184821b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:06.001821 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf11d7-fd00-4eef-b31c-776d7184821b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:06.041940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3b336-0668-44f1-8a1f-8bec72698588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:06.043201 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3b336-0668-44f1-8a1f-8bec72698588 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:06.043798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3b336-0668-44f1-8a1f-8bec72698588 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:06.044366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3b336-0668-44f1-8a1f-8bec72698588 None admin] PATH_INFO: `/v3/users/b519d0db93834e6491f62b7d72ce0dc6` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:06.046258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3b336-0668-44f1-8a1f-8bec72698588 None admin] RBAC: Authorizing `identity:get_user(user_id=b519d0db93834e6491f62b7d72ce0dc6)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:06.048192 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3b336-0668-44f1-8a1f-8bec72698588 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:06.050899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 122/232] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:05 2022] GET /identity/v3/users/b519d0db93834e6491f62b7d72ce0dc6 => generated 284 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:06.059208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e757d5-4330-4a69-9b01-d4fdbc570539 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:06.203241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e757d5-4330-4a69-9b01-d4fdbc570539 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:06.236674 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:06.246451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e6418-d8ef-4f30-85bb-7e90d1bb24df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:06.247139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e6418-d8ef-4f30-85bb-7e90d1bb24df None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:06.247424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e6418-d8ef-4f30-85bb-7e90d1bb24df None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:06.247700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e6418-d8ef-4f30-85bb-7e90d1bb24df None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:06.248884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e6418-d8ef-4f30-85bb-7e90d1bb24df None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:06.250353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e6418-d8ef-4f30-85bb-7e90d1bb24df None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:06.252433 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 111/233] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:06 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:06.258091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3b336-0668-44f1-8a1f-8bec72698588 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:06.361292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:06.429351 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3b336-0668-44f1-8a1f-8bec72698588 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:06.478777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad0981-f13c-41c1-b4c0-40f9eff83396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:06.479467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad0981-f13c-41c1-b4c0-40f9eff83396 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:06.479749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad0981-f13c-41c1-b4c0-40f9eff83396 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:06.480029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad0981-f13c-41c1-b4c0-40f9eff83396 None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f/users/b519d0db93834e6491f62b7d72ce0dc6/roles/57221d717a454c7fae4f2df1de9dbe5b` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:06.482262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad0981-f13c-41c1-b4c0-40f9eff83396 None admin] RBAC: Authorizing `identity:create_grant(project_id=ebb7ad89c9734ecbbd41c0c2de27069f, user_id=b519d0db93834e6491f62b7d72ce0dc6, role_id=57221d717a454c7fae4f2df1de9dbe5b)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:06.483788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad0981-f13c-41c1-b4c0-40f9eff83396 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:06.509432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 123/234] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:00:06 2022] PUT /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f/users/b519d0db93834e6491f62b7d72ce0dc6/roles/57221d717a454c7fae4f2df1de9dbe5b => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:07.146040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac51283-c6b8-491c-90f0-abad831455af None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:07.146564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac51283-c6b8-491c-90f0-abad831455af None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:07.146564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac51283-c6b8-491c-90f0-abad831455af None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:07.148209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 112/235] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:07.158164 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cec5a-09eb-4b6a-873e-91743095c680 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:07.158745 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cec5a-09eb-4b6a-873e-91743095c680 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:07.159019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cec5a-09eb-4b6a-873e-91743095c680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:07.170349 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48501ba7-b703-49ce-948c-fba144999c7e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:07.170635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48501ba7-b703-49ce-948c-fba144999c7e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:07.171357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48501ba7-b703-49ce-948c-fba144999c7e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:07.172871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 113/236] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:07.185951 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287c787-99a2-4b2f-abd5-f0bfd7d0bba4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:07.186257 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287c787-99a2-4b2f-abd5-f0bfd7d0bba4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:07.186512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287c787-99a2-4b2f-abd5-f0bfd7d0bba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:07.188610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c30cec5a-09eb-4b6a-873e-91743095c680 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:07.189932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c30cec5a-09eb-4b6a-873e-91743095c680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:07.219725 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d287c787-99a2-4b2f-abd5-f0bfd7d0bba4 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:07.220990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d287c787-99a2-4b2f-abd5-f0bfd7d0bba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:07.309217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 124/237] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:07.317066 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 114/238] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:07.355219 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30cec5a-09eb-4b6a-873e-91743095c680 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:07.360722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287c787-99a2-4b2f-abd5-f0bfd7d0bba4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:07.487803 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:07.531079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287c787-99a2-4b2f-abd5-f0bfd7d0bba4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:07.597968 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30cec5a-09eb-4b6a-873e-91743095c680 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:07.622153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:07.646929 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e924c4-2b49-42d5-bf0e-79ed28521f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:07.647592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e924c4-2b49-42d5-bf0e-79ed28521f1e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:07.647945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e924c4-2b49-42d5-bf0e-79ed28521f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:07.648253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e924c4-2b49-42d5-bf0e-79ed28521f1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:07.649883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e924c4-2b49-42d5-bf0e-79ed28521f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:07.651449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e924c4-2b49-42d5-bf0e-79ed28521f1e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:07.653069 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 125/239] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:07 2022] GET /identity/v3/domains/default => generated 200 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:07.653850 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17613111-5a19-42f0-b3db-52e03c84d225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:07.654569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17613111-5a19-42f0-b3db-52e03c84d225 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:07.654875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17613111-5a19-42f0-b3db-52e03c84d225 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:07.655149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17613111-5a19-42f0-b3db-52e03c84d225 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:07.660107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e924c4-2b49-42d5-bf0e-79ed28521f1e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:07.677965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17613111-5a19-42f0-b3db-52e03c84d225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-17613111-5a19-42f0-b3db-52e03c84d225 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:07.686502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:07.688057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:07.689451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:07.689451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:07.689451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:07.689451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:07.689451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:07.689451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:07.689451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:07.689451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:07.689451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:07.689451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:07.689451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:00:07.689451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:07.689451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 115/240] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:07.696456 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17613111-5a19-42f0-b3db-52e03c84d225 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:07.879787 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e924c4-2b49-42d5-bf0e-79ed28521f1e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:07.892040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17613111-5a19-42f0-b3db-52e03c84d225 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:07.901490 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:07.922975 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308e833a-507d-41d8-89c5-264c4f0ec487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:07.924232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e833a-507d-41d8-89c5-264c4f0ec487 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:07.925169 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e833a-507d-41d8-89c5-264c4f0ec487 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:07.925855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308e833a-507d-41d8-89c5-264c4f0ec487 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:07.927483 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e833a-507d-41d8-89c5-264c4f0ec487 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:07.929455 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308e833a-507d-41d8-89c5-264c4f0ec487 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:07.930903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ae7fea-73ba-4d64-833a-36b2c9feb580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:07.931643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae7fea-73ba-4d64-833a-36b2c9feb580 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:07.932026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae7fea-73ba-4d64-833a-36b2c9feb580 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:07.932517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae7fea-73ba-4d64-833a-36b2c9feb580 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:07.933990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae7fea-73ba-4d64-833a-36b2c9feb580 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:07.934728 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae7fea-73ba-4d64-833a-36b2c9feb580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:07.936446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ae7fea-73ba-4d64-833a-36b2c9feb580 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:07.948721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 116/241] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:07 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:07.950290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582716a-03c5-4c17-aafd-d7a536d22e00 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:07.950712 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582716a-03c5-4c17-aafd-d7a536d22e00 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:07.951106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582716a-03c5-4c17-aafd-d7a536d22e00 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:07.958778 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 117/242] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:07 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:07.959677 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 126/243] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:00:07 2022] POST /identity/v3/users => generated 294 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:00:07.961494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582716a-03c5-4c17-aafd-d7a536d22e00 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:07.963804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790a779-e2fc-43bd-8693-d5be014b9e13 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:07.964262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790a779-e2fc-43bd-8693-d5be014b9e13 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:07.964663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790a779-e2fc-43bd-8693-d5be014b9e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:07.989758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4790a779-e2fc-43bd-8693-d5be014b9e13 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:07.991511 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4790a779-e2fc-43bd-8693-d5be014b9e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:08.081752 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 127/244] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:08.122941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790a779-e2fc-43bd-8693-d5be014b9e13 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:08.142845 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:08.227356 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582716a-03c5-4c17-aafd-d7a536d22e00 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:08.277663 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19258984-76b6-49f8-bb72-e630eb8d01a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:08.278882 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19258984-76b6-49f8-bb72-e630eb8d01a6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:08.279169 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19258984-76b6-49f8-bb72-e630eb8d01a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:08.279453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19258984-76b6-49f8-bb72-e630eb8d01a6 None admin] PATH_INFO: `/v3/users/0bbb19525e8f45e6bc60406c4972e45e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:08.305771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19258984-76b6-49f8-bb72-e630eb8d01a6 None admin] RBAC: Authorizing `identity:get_user(user_id=0bbb19525e8f45e6bc60406c4972e45e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:08.305994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790a779-e2fc-43bd-8693-d5be014b9e13 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:08.307233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19258984-76b6-49f8-bb72-e630eb8d01a6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:08.308361 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:08.310009 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 118/245] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:07 2022] GET /identity/v3/users/0bbb19525e8f45e6bc60406c4972e45e => generated 298 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:08.316393 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19258984-76b6-49f8-bb72-e630eb8d01a6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:08.354250 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a3286-1701-4388-b96e-f86e3219c92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:08.355206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a3286-1701-4388-b96e-f86e3219c92d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:08.355485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a3286-1701-4388-b96e-f86e3219c92d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:08.355829 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a3286-1701-4388-b96e-f86e3219c92d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:08.371017 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a3286-1701-4388-b96e-f86e3219c92d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-a33a3286-1701-4388-b96e-f86e3219c92d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:08.379233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:08.380173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:08.381622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:08.381622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:08.381622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:08.381622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:08.381622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:08.381622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:08.381622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:08.381622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:08.381622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:08.381622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:08.381622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:08.381622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:08.381622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 128/246] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:08.389272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a3286-1701-4388-b96e-f86e3219c92d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:08.463931 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19258984-76b6-49f8-bb72-e630eb8d01a6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:08.475813 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:08.526162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bdb9d-b62b-4163-820a-6dd566e56ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:08.526939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bdb9d-b62b-4163-820a-6dd566e56ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:08.527231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bdb9d-b62b-4163-820a-6dd566e56ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:08.527563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bdb9d-b62b-4163-820a-6dd566e56ef8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:08.534209 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a3286-1701-4388-b96e-f86e3219c92d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:08.582045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44dbbef-e5b8-4c6c-be51-6dae52934621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:08.582768 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44dbbef-e5b8-4c6c-be51-6dae52934621 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:08.583050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44dbbef-e5b8-4c6c-be51-6dae52934621 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:08.583370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44dbbef-e5b8-4c6c-be51-6dae52934621 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:08.584166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44dbbef-e5b8-4c6c-be51-6dae52934621 None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:08.584471 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44dbbef-e5b8-4c6c-be51-6dae52934621 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:08.586053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44dbbef-e5b8-4c6c-be51-6dae52934621 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:08.597654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 129/247] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:08 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:08.600257 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b04b2-70df-4448-a99b-cd16326a88cb None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:08.600548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b04b2-70df-4448-a99b-cd16326a88cb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:08.600823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b04b2-70df-4448-a99b-cd16326a88cb None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:08.602330 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 130/248] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:00:08.607745 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b04b2-70df-4448-a99b-cd16326a88cb None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:08.627707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bdb9d-b62b-4163-820a-6dd566e56ef8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:08.629350 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bdb9d-b62b-4163-820a-6dd566e56ef8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:08.647244 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:08.781914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b04b2-70df-4448-a99b-cd16326a88cb None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:08.823199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db33429-02a0-4b88-9cb2-9af4c611da49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:08.823858 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db33429-02a0-4b88-9cb2-9af4c611da49 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:08.824138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db33429-02a0-4b88-9cb2-9af4c611da49 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:08.824414 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db33429-02a0-4b88-9cb2-9af4c611da49 None admin] PATH_INFO: `/v3/users/d30378f810a1497f94191b23bd742912` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:08.825620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db33429-02a0-4b88-9cb2-9af4c611da49 None admin] RBAC: Authorizing `identity:get_user(user_id=d30378f810a1497f94191b23bd742912)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:08.827568 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db33429-02a0-4b88-9cb2-9af4c611da49 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:08.830016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 131/249] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:08 2022] GET /identity/v3/users/d30378f810a1497f94191b23bd742912 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:08.832311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca3046-ecc0-4f26-af1c-2f521ac1d014 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:08.832741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca3046-ecc0-4f26-af1c-2f521ac1d014 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:08.833173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca3046-ecc0-4f26-af1c-2f521ac1d014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:08.841597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 119/250] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:08 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:08.842556 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bdb9d-b62b-4163-820a-6dd566e56ef8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:08.858427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1fca3046-ecc0-4f26-af1c-2f521ac1d014 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:08.859700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1fca3046-ecc0-4f26-af1c-2f521ac1d014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:08.863025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:08.948398 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 132/251] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:08.949379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca3046-ecc0-4f26-af1c-2f521ac1d014 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:08.996493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:09.040271 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bdb9d-b62b-4163-820a-6dd566e56ef8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:09.078920 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3ffb7-be7b-40cc-848c-fbcc64a1c821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:09.079579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3ffb7-be7b-40cc-848c-fbcc64a1c821 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.079858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3ffb7-be7b-40cc-848c-fbcc64a1c821 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.080175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3ffb7-be7b-40cc-848c-fbcc64a1c821 None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.081999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3ffb7-be7b-40cc-848c-fbcc64a1c821 None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:09.083798 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3ffb7-be7b-40cc-848c-fbcc64a1c821 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:09.088975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 120/252] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:08 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:09.089858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3ffb7-be7b-40cc-848c-fbcc64a1c821 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:09.177306 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca3046-ecc0-4f26-af1c-2f521ac1d014 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:09.182346 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:09.210079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3ffb7-be7b-40cc-848c-fbcc64a1c821 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:09.250022 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386e934-d274-4d77-9950-9cbe69511706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:09.250765 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d74bca-6c78-4f4d-9285-b29810b7d62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:09.250858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386e934-d274-4d77-9950-9cbe69511706 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.250929 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d74bca-6c78-4f4d-9285-b29810b7d62f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.251029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386e934-d274-4d77-9950-9cbe69511706 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.251287 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d74bca-6c78-4f4d-9285-b29810b7d62f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.251382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386e934-d274-4d77-9950-9cbe69511706 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.251451 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d74bca-6c78-4f4d-9285-b29810b7d62f None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.252584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d74bca-6c78-4f4d-9285-b29810b7d62f None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:09.254842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d74bca-6c78-4f4d-9285-b29810b7d62f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:09.256556 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 133/253] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:08 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:09.257654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d74bca-6c78-4f4d-9285-b29810b7d62f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:09.263228 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386e934-d274-4d77-9950-9cbe69511706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0386e934-d274-4d77-9950-9cbe69511706 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:09.268708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:09.269835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:09.271073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:09.271073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:09.271073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:09.271073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:09.271073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:09.271073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:09.271073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:09.271073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:09.271073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:09.271073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:09.271073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:09.271073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:09.271073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 121/254] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:00:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:09.271984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbdb93-b5d3-4912-b6f3-67c08c846e61 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.272052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbdb93-b5d3-4912-b6f3-67c08c846e61 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.272329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbdb93-b5d3-4912-b6f3-67c08c846e61 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.274178 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 122/255] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:09.274924 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbdb93-b5d3-4912-b6f3-67c08c846e61 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:09.385978 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:09.389692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbdb93-b5d3-4912-b6f3-67c08c846e61 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:09.397489 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d74bca-6c78-4f4d-9285-b29810b7d62f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:09.429830 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e5114-999e-4796-95a8-7e5a8f1575a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:09.430995 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e5114-999e-4796-95a8-7e5a8f1575a5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.431416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e5114-999e-4796-95a8-7e5a8f1575a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.431827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e5114-999e-4796-95a8-7e5a8f1575a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.432871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e5114-999e-4796-95a8-7e5a8f1575a5 None admin] RBAC: Adding query filter params (role.id=471fd358281343cba97902b70b416f32, scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f, user.id=0bbb19525e8f45e6bc60406c4972e45e) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:09.433438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e5114-999e-4796-95a8-7e5a8f1575a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:09.435115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e5114-999e-4796-95a8-7e5a8f1575a5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:09.437364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:09.439798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3abbd-30e9-4465-a43a-8525ea4a745b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:09.440570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3abbd-30e9-4465-a43a-8525ea4a745b None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.441581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3abbd-30e9-4465-a43a-8525ea4a745b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.441950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3abbd-30e9-4465-a43a-8525ea4a745b None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f/users/d30378f810a1497f94191b23bd742912/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.443781 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 123/256] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:09 2022] GET /identity/v3/role_assignments?user.id=0bbb19525e8f45e6bc60406c4972e45e&scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f&role.id=471fd358281343cba97902b70b416f32 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:09.444548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3abbd-30e9-4465-a43a-8525ea4a745b None admin] RBAC: Authorizing `identity:create_grant(project_id=ebb7ad89c9734ecbbd41c0c2de27069f, user_id=d30378f810a1497f94191b23bd742912, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:09.445148 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e5114-999e-4796-95a8-7e5a8f1575a5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:09.446432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3abbd-30e9-4465-a43a-8525ea4a745b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:09.465179 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 134/257] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:00:09 2022] PUT /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f/users/d30378f810a1497f94191b23bd742912/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:09.466976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff5ae2-21a3-46f6-a498-162d96ecd0e9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.467273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff5ae2-21a3-46f6-a498-162d96ecd0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.467685 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff5ae2-21a3-46f6-a498-162d96ecd0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.491699 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d4ff5ae2-21a3-46f6-a498-162d96ecd0e9 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:09.492938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d4ff5ae2-21a3-46f6-a498-162d96ecd0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:09.572936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 135/258] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:09.589389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e5114-999e-4796-95a8-7e5a8f1575a5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:09.652510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4869ff7-1efa-43fa-a2af-a051922ea49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:09.653068 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4869ff7-1efa-43fa-a2af-a051922ea49b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.653424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4869ff7-1efa-43fa-a2af-a051922ea49b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.653703 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4869ff7-1efa-43fa-a2af-a051922ea49b None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.654505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4869ff7-1efa-43fa-a2af-a051922ea49b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:09.654842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4869ff7-1efa-43fa-a2af-a051922ea49b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:09.656583 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4869ff7-1efa-43fa-a2af-a051922ea49b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:09.667624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 124/259] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:09 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:09.668617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff5ae2-21a3-46f6-a498-162d96ecd0e9 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:09.674075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4869ff7-1efa-43fa-a2af-a051922ea49b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:09.747220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:09.795495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4869ff7-1efa-43fa-a2af-a051922ea49b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:09.835531 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad772d-9e7b-49c6-a6e4-3db3460d059a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:09.837214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad772d-9e7b-49c6-a6e4-3db3460d059a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.837569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad772d-9e7b-49c6-a6e4-3db3460d059a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.837910 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad772d-9e7b-49c6-a6e4-3db3460d059a None admin] PATH_INFO: `/v3/users/b519d0db93834e6491f62b7d72ce0dc6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.839699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad772d-9e7b-49c6-a6e4-3db3460d059a None admin] RBAC: Authorizing `identity:get_user(user_id=b519d0db93834e6491f62b7d72ce0dc6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:09.841261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad772d-9e7b-49c6-a6e4-3db3460d059a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:09.843142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 125/260] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:09 2022] GET /identity/v3/users/b519d0db93834e6491f62b7d72ce0dc6 => generated 284 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:09.847255 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:09.851939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad772d-9e7b-49c6-a6e4-3db3460d059a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:09.874174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff5ae2-21a3-46f6-a498-162d96ecd0e9 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:09.923462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6250a-a694-408b-b294-f5644c5d4fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:09.924062 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6250a-a694-408b-b294-f5644c5d4fea None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:09.924373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6250a-a694-408b-b294-f5644c5d4fea None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:09.924608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6250a-a694-408b-b294-f5644c5d4fea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:09.941545 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6250a-a694-408b-b294-f5644c5d4fea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-ecb6250a-a694-408b-b294-f5644c5d4fea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:09.947536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:09.948500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:09.949312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:09.949312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:09.949312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:09.949312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:09.949312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:09.949312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:09.949312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:09.949312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:09.949312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:09.949312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:09.949312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:09.949312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:09.949312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 136/261] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:00:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:09.963893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6250a-a694-408b-b294-f5644c5d4fea None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:09.975740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad772d-9e7b-49c6-a6e4-3db3460d059a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:10.004499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:10.014129 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3cbde-256c-4d8f-83b6-d38f1124371f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:10.015014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3cbde-256c-4d8f-83b6-d38f1124371f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:10.015420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3cbde-256c-4d8f-83b6-d38f1124371f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:10.015857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3cbde-256c-4d8f-83b6-d38f1124371f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:10.103548 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3cbde-256c-4d8f-83b6-d38f1124371f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:10.105301 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3cbde-256c-4d8f-83b6-d38f1124371f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:10.128485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6250a-a694-408b-b294-f5644c5d4fea None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:10.175833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513e755-7bf3-4aae-b629-117361024a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:10.176380 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513e755-7bf3-4aae-b629-117361024a70 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:10.176661 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513e755-7bf3-4aae-b629-117361024a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:10.176934 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513e755-7bf3-4aae-b629-117361024a70 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:10.177722 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513e755-7bf3-4aae-b629-117361024a70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:10.181456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513e755-7bf3-4aae-b629-117361024a70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:10.181456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513e755-7bf3-4aae-b629-117361024a70 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:10.190039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 137/262] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:09 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:10.197046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513e755-7bf3-4aae-b629-117361024a70 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:10.215145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:10.342447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 126/263] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:09 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:00:10.350327 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3cbde-256c-4d8f-83b6-d38f1124371f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:10.396239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:10.408533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513e755-7bf3-4aae-b629-117361024a70 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:10.461539 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b14d1a-7e5e-42da-9dc4-119b8009b0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:10.462248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b14d1a-7e5e-42da-9dc4-119b8009b0ae None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:10.462535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b14d1a-7e5e-42da-9dc4-119b8009b0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:10.462837 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b14d1a-7e5e-42da-9dc4-119b8009b0ae None admin] PATH_INFO: `/v3/users/b519d0db93834e6491f62b7d72ce0dc6` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:10.464784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b14d1a-7e5e-42da-9dc4-119b8009b0ae None admin] RBAC: Authorizing `identity:get_user(user_id=b519d0db93834e6491f62b7d72ce0dc6)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:10.466317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b14d1a-7e5e-42da-9dc4-119b8009b0ae None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:10.468374 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 138/264] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:10 2022] GET /identity/v3/users/b519d0db93834e6491f62b7d72ce0dc6 => generated 284 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:10.473926 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b14d1a-7e5e-42da-9dc4-119b8009b0ae None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:10.546155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3cbde-256c-4d8f-83b6-d38f1124371f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:10.587245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:10.624788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b14d1a-7e5e-42da-9dc4-119b8009b0ae None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:10.631605 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c08f8-7669-4d0c-8774-4f67c91148a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:10.633366 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c08f8-7669-4d0c-8774-4f67c91148a7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:10.634052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c08f8-7669-4d0c-8774-4f67c91148a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:10.634735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c08f8-7669-4d0c-8774-4f67c91148a7 None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:10.636680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c08f8-7669-4d0c-8774-4f67c91148a7 None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:10.639110 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c08f8-7669-4d0c-8774-4f67c91148a7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:10.641859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 127/265] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:10 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:10.653590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c08f8-7669-4d0c-8774-4f67c91148a7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:10.667290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edf0b2-6950-479f-86e5-8fd3f1ba6413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:10.668061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edf0b2-6950-479f-86e5-8fd3f1ba6413 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:10.668396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edf0b2-6950-479f-86e5-8fd3f1ba6413 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:10.668741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edf0b2-6950-479f-86e5-8fd3f1ba6413 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:10.766533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02edf0b2-6950-479f-86e5-8fd3f1ba6413 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:10.768213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02edf0b2-6950-479f-86e5-8fd3f1ba6413 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:10.907296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:10.963867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 139/266] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:10 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:00:10.968295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:10.971047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edf0b2-6950-479f-86e5-8fd3f1ba6413 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:10.984908 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c08f8-7669-4d0c-8774-4f67c91148a7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:11.046245 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0feeb43-1faa-4bfb-9a0b-8fd10b3f4497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:11.046803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0feeb43-1faa-4bfb-9a0b-8fd10b3f4497 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:11.047091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0feeb43-1faa-4bfb-9a0b-8fd10b3f4497 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:11.047372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0feeb43-1faa-4bfb-9a0b-8fd10b3f4497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:11.048173 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0feeb43-1faa-4bfb-9a0b-8fd10b3f4497 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f, user.id=b519d0db93834e6491f62b7d72ce0dc6) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:11.048487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0feeb43-1faa-4bfb-9a0b-8fd10b3f4497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:11.049963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0feeb43-1faa-4bfb-9a0b-8fd10b3f4497 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:11.060302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 128/267] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:10 2022] GET /identity/v3/role_assignments?user.id=b519d0db93834e6491f62b7d72ce0dc6&scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 610 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:11.136182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edf0b2-6950-479f-86e5-8fd3f1ba6413 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:11.153098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3a14e-8d3e-496e-b704-1a49d47b9679 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:11.153406 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3a14e-8d3e-496e-b704-1a49d47b9679 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:11.153652 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3a14e-8d3e-496e-b704-1a49d47b9679 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:11.155942 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 129/268] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:00:11.163602 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f808e-14b6-43a6-9385-47b6b549b944 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:11.165533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f808e-14b6-43a6-9385-47b6b549b944 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:11.165844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f808e-14b6-43a6-9385-47b6b549b944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:11.191117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-205f808e-14b6-43a6-9385-47b6b549b944 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:11.192952 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-205f808e-14b6-43a6-9385-47b6b549b944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:11.222262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cc275-2c9d-4ff3-a91e-523fdb0065de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:11.222964 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cc275-2c9d-4ff3-a91e-523fdb0065de None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:11.223248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cc275-2c9d-4ff3-a91e-523fdb0065de None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:11.223525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cc275-2c9d-4ff3-a91e-523fdb0065de None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:11.225036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cc275-2c9d-4ff3-a91e-523fdb0065de None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:11.227118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cc275-2c9d-4ff3-a91e-523fdb0065de None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:11.231703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 140/269] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:10 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:11.236458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cc275-2c9d-4ff3-a91e-523fdb0065de None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:11.286777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:11.351884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cc275-2c9d-4ff3-a91e-523fdb0065de None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:11.358680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 130/270] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:00:11.405516 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9bef7-1103-4f55-b119-08f955acb22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:11.406398 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9bef7-1103-4f55-b119-08f955acb22f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:11.406868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9bef7-1103-4f55-b119-08f955acb22f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:11.407284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9bef7-1103-4f55-b119-08f955acb22f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:11.408223 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9bef7-1103-4f55-b119-08f955acb22f None admin] RBAC: Adding query filter params (role.id=57221d717a454c7fae4f2df1de9dbe5b, scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f, user.id=b519d0db93834e6491f62b7d72ce0dc6) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:11.408705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9bef7-1103-4f55-b119-08f955acb22f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:11.410788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d9bef7-1103-4f55-b119-08f955acb22f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:11.422786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 141/271] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:11 2022] GET /identity/v3/role_assignments?user.id=b519d0db93834e6491f62b7d72ce0dc6&scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f&role.id=57221d717a454c7fae4f2df1de9dbe5b => generated 610 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:11.428578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f808e-14b6-43a6-9385-47b6b549b944 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:11.590267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:11.600261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f808e-14b6-43a6-9385-47b6b549b944 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:11.636645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16ee96-3839-4dab-a8d0-ca9a4127dc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:11.637221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16ee96-3839-4dab-a8d0-ca9a4127dc3c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:11.637506 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16ee96-3839-4dab-a8d0-ca9a4127dc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:11.637788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16ee96-3839-4dab-a8d0-ca9a4127dc3c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:11.653507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16ee96-3839-4dab-a8d0-ca9a4127dc3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-ae16ee96-3839-4dab-a8d0-ca9a4127dc3c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:11.659700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:11.661279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:11.664424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:11.664424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:11.664424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:11.664424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:11.664424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:11.664424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:11.664424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:11.664424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:11.664424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:11.664424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:11.664424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:11.664424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:11.664424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 131/272] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:11.680711 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9bef7-1103-4f55-b119-08f955acb22f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:11.844356 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9bef7-1103-4f55-b119-08f955acb22f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:11.853286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:11.864574 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fe667-7d3d-4c6c-a2d4-f665ae90e1b4 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:11.864870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fe667-7d3d-4c6c-a2d4-f665ae90e1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:11.865137 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fe667-7d3d-4c6c-a2d4-f665ae90e1b4 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:11.867601 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 132/273] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:11.871933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5324596-83b9-4988-9773-bce1d0ff56d7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:11.872220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5324596-83b9-4988-9773-bce1d0ff56d7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:11.872494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5324596-83b9-4988-9773-bce1d0ff56d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:11.886748 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4398b-a635-4f26-82ed-61fee546b38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:11.887746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4398b-a635-4f26-82ed-61fee546b38d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:11.888235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4398b-a635-4f26-82ed-61fee546b38d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:11.888697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4398b-a635-4f26-82ed-61fee546b38d None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:11.889610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4398b-a635-4f26-82ed-61fee546b38d None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:11.890128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4398b-a635-4f26-82ed-61fee546b38d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:11.892077 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d4398b-a635-4f26-82ed-61fee546b38d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:11.902490 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 142/274] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:11 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:11.910339 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e5324596-83b9-4988-9773-bce1d0ff56d7 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:11.911590 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4398b-a635-4f26-82ed-61fee546b38d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:11.913315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e5324596-83b9-4988-9773-bce1d0ff56d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:12.004302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 133/275] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:12.030557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5324596-83b9-4988-9773-bce1d0ff56d7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:12.042118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4398b-a635-4f26-82ed-61fee546b38d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:12.077835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af02b67-cbf4-482c-8ea6-8600cb9f8635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:12.078554 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af02b67-cbf4-482c-8ea6-8600cb9f8635 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:12.078884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af02b67-cbf4-482c-8ea6-8600cb9f8635 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:12.079230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af02b67-cbf4-482c-8ea6-8600cb9f8635 None admin] PATH_INFO: `/v3/users/00aad4fb761745499a6673f7b2bb88dd` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:12.095667 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af02b67-cbf4-482c-8ea6-8600cb9f8635 None admin] RBAC: Authorizing `identity:get_user(user_id=00aad4fb761745499a6673f7b2bb88dd)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:12.097200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af02b67-cbf4-482c-8ea6-8600cb9f8635 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:12.098927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 143/276] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:11 2022] GET /identity/v3/users/00aad4fb761745499a6673f7b2bb88dd => generated 294 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:12.104974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af02b67-cbf4-482c-8ea6-8600cb9f8635 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:12.129190 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:12.132308 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:12.155276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5324596-83b9-4988-9773-bce1d0ff56d7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:12.194489 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6bc6b-dc5d-4912-b310-79e7c614e0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:12.196131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6bc6b-dc5d-4912-b310-79e7c614e0dc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:12.196732 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6bc6b-dc5d-4912-b310-79e7c614e0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:12.197239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6bc6b-dc5d-4912-b310-79e7c614e0dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:12.198611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6bc6b-dc5d-4912-b310-79e7c614e0dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:12.200525 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6bc6b-dc5d-4912-b310-79e7c614e0dc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:12.206984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 134/277] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:12 2022] GET /identity/v3/domains/default => generated 200 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:12.208882 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6bc6b-dc5d-4912-b310-79e7c614e0dc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:12.217603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af02b67-cbf4-482c-8ea6-8600cb9f8635 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:12.255167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d29c2d-d8e0-4409-aa92-0dbfccf6a746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:12.255794 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d29c2d-d8e0-4409-aa92-0dbfccf6a746 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:12.256078 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d29c2d-d8e0-4409-aa92-0dbfccf6a746 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:12.256354 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d29c2d-d8e0-4409-aa92-0dbfccf6a746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:12.257154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d29c2d-d8e0-4409-aa92-0dbfccf6a746 None admin] RBAC: Adding query filter params (role.id=d1f6973ce11a4644b649b38f8b2df173, user.id=00aad4fb761745499a6673f7b2bb88dd, scope.system=all) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:12.257582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d29c2d-d8e0-4409-aa92-0dbfccf6a746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:12.259566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d29c2d-d8e0-4409-aa92-0dbfccf6a746 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:12.265054 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 144/278] 10.222.0.100 () {56 vars in 1313 bytes} [Mon Aug 1 15:00:12 2022] GET /identity/v3/role_assignments?user.id=00aad4fb761745499a6673f7b2bb88dd&scope.system=all&role.id=d1f6973ce11a4644b649b38f8b2df173 => generated 227 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:12.333233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6bc6b-dc5d-4912-b310-79e7c614e0dc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:12.372518 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a516b-ca65-40c6-9a85-84c44dd7621a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:12.374068 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a516b-ca65-40c6-9a85-84c44dd7621a None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:12.374640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a516b-ca65-40c6-9a85-84c44dd7621a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:12.376687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cff495-26da-43dd-a6a5-1c135b1ca73e None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:12.377185 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a516b-ca65-40c6-9a85-84c44dd7621a None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:12.377824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cff495-26da-43dd-a6a5-1c135b1ca73e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:12.378289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a516b-ca65-40c6-9a85-84c44dd7621a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:12.378610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cff495-26da-43dd-a6a5-1c135b1ca73e None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:12.380384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a516b-ca65-40c6-9a85-84c44dd7621a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:12.380907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 145/279] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:12 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:00:12.388063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f80ed-104c-4b87-b0b1-50c2c353c8b0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:12.388583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f80ed-104c-4b87-b0b1-50c2c353c8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:12.389178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f80ed-104c-4b87-b0b1-50c2c353c8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:12.410224 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 135/280] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:00:12 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:00:12.419707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3f2f80ed-104c-4b87-b0b1-50c2c353c8b0 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:12.421156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3f2f80ed-104c-4b87-b0b1-50c2c353c8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:12.510732 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 146/281] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:12.532671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a516b-ca65-40c6-9a85-84c44dd7621a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:12.547496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:12.574145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6534aa1-f5af-4033-868f-dffe11165f05 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:12.574499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6534aa1-f5af-4033-868f-dffe11165f05 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:12.574736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6534aa1-f5af-4033-868f-dffe11165f05 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:12.576377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 147/282] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:12.581939 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b50a71-7743-46c2-b6b0-cd049b579bcc None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:12.582347 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b50a71-7743-46c2-b6b0-cd049b579bcc None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:12.582473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b50a71-7743-46c2-b6b0-cd049b579bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:12.602470 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b6b50a71-7743-46c2-b6b0-cd049b579bcc None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:12.603705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b6b50a71-7743-46c2-b6b0-cd049b579bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:12.627056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:12.678322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a516b-ca65-40c6-9a85-84c44dd7621a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:12.708056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 148/283] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:00:12.721326 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9a363-9c0c-4fd1-8362-1a3ed090781e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:12.722423 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9a363-9c0c-4fd1-8362-1a3ed090781e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:12.722757 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9a363-9c0c-4fd1-8362-1a3ed090781e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:12.723043 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a9a363-9c0c-4fd1-8362-1a3ed090781e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:12.731915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b50a71-7743-46c2-b6b0-cd049b579bcc None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:12.737412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a9a363-9c0c-4fd1-8362-1a3ed090781e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-04a9a363-9c0c-4fd1-8362-1a3ed090781e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:12.744760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:12.745685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:12.747134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:12.747134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:12.747134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:12.747134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:12.747134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:12.747134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:12.747134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:12.747134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:12.747134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:12.747134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:12.747134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:00:12.747134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:12.747134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 136/284] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:12.751879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9a363-9c0c-4fd1-8362-1a3ed090781e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:12.830530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:12.870777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b50a71-7743-46c2-b6b0-cd049b579bcc None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:12.883387 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a9a363-9c0c-4fd1-8362-1a3ed090781e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:12.917600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c6948-0538-4086-a924-153bd1183285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:12.918564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c6948-0538-4086-a924-153bd1183285 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:12.918876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c6948-0538-4086-a924-153bd1183285 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:12.919448 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c6948-0538-4086-a924-153bd1183285 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:12.929537 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58257eb7-1f45-477c-abcc-70729d3e73ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:12.930331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58257eb7-1f45-477c-abcc-70729d3e73ec None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:12.930805 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58257eb7-1f45-477c-abcc-70729d3e73ec None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:12.931156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58257eb7-1f45-477c-abcc-70729d3e73ec None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:12.932250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58257eb7-1f45-477c-abcc-70729d3e73ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:12.932631 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58257eb7-1f45-477c-abcc-70729d3e73ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:12.933626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c6948-0538-4086-a924-153bd1183285 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:12.934184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58257eb7-1f45-477c-abcc-70729d3e73ec None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-ba5c6948-0538-4086-a924-153bd1183285 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:12.939754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:12.941001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:12.942001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:12.942001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:12.942001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:12.942001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:12.942001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:12.942001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:12.942001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:12.942001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:12.942001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:12.942001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:12.942001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:12.942001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:12.942001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 149/285] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:12.949976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c6948-0538-4086-a924-153bd1183285 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:12.950374 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 137/286] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:12 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:12.959652 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58257eb7-1f45-477c-abcc-70729d3e73ec None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:12.968183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:13.068425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c6948-0538-4086-a924-153bd1183285 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:13.106522 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799326ca-9254-4f56-b707-62842ec5d289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:13.107183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799326ca-9254-4f56-b707-62842ec5d289 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:13.107465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799326ca-9254-4f56-b707-62842ec5d289 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:13.107747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799326ca-9254-4f56-b707-62842ec5d289 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:13.108570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799326ca-9254-4f56-b707-62842ec5d289 None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:13.108878 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799326ca-9254-4f56-b707-62842ec5d289 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:13.110356 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799326ca-9254-4f56-b707-62842ec5d289 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:13.124822 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 150/287] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:12 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:13.126463 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799326ca-9254-4f56-b707-62842ec5d289 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:13.205725 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58257eb7-1f45-477c-abcc-70729d3e73ec None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:13.240951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:13.267220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:13.278002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c6fab-a6f5-4fb8-81cd-9b8f93ebf603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:13.279271 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c6fab-a6f5-4fb8-81cd-9b8f93ebf603 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:13.279816 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c6fab-a6f5-4fb8-81cd-9b8f93ebf603 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:13.280306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c6fab-a6f5-4fb8-81cd-9b8f93ebf603 None admin] PATH_INFO: `/v3/users/0bbb19525e8f45e6bc60406c4972e45e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:13.282623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c6fab-a6f5-4fb8-81cd-9b8f93ebf603 None admin] RBAC: Authorizing `identity:get_user(user_id=0bbb19525e8f45e6bc60406c4972e45e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:13.285655 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c6fab-a6f5-4fb8-81cd-9b8f93ebf603 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:13.290186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 138/288] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:12 2022] GET /identity/v3/users/0bbb19525e8f45e6bc60406c4972e45e => generated 298 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:13.296408 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c6fab-a6f5-4fb8-81cd-9b8f93ebf603 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:13.334686 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799326ca-9254-4f56-b707-62842ec5d289 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:13.414639 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e93c4-0b9a-4075-82b9-cd6dd088acb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:13.415313 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e93c4-0b9a-4075-82b9-cd6dd088acb2 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:13.415592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e93c4-0b9a-4075-82b9-cd6dd088acb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:13.415865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e93c4-0b9a-4075-82b9-cd6dd088acb2 None admin] PATH_INFO: `/v3/users/d30378f810a1497f94191b23bd742912` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:13.417798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e93c4-0b9a-4075-82b9-cd6dd088acb2 None admin] RBAC: Authorizing `identity:get_user(user_id=d30378f810a1497f94191b23bd742912)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:13.420161 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e93c4-0b9a-4075-82b9-cd6dd088acb2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:13.430692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 151/289] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:13 2022] GET /identity/v3/users/d30378f810a1497f94191b23bd742912 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:13.430692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e93c4-0b9a-4075-82b9-cd6dd088acb2 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:13.482121 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c6fab-a6f5-4fb8-81cd-9b8f93ebf603 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:13.557166 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26060517-e131-4cde-804e-6834aa81f6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:13.557833 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26060517-e131-4cde-804e-6834aa81f6ee None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:13.558124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26060517-e131-4cde-804e-6834aa81f6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:13.558407 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26060517-e131-4cde-804e-6834aa81f6ee None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:13.563556 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26060517-e131-4cde-804e-6834aa81f6ee None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:13.565114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26060517-e131-4cde-804e-6834aa81f6ee None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:13.567281 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 139/290] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:13 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:13.574205 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26060517-e131-4cde-804e-6834aa81f6ee None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:13.596262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e93c4-0b9a-4075-82b9-cd6dd088acb2 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:13.669020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fad97-99e2-4cc6-8525-9900d9fee37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:13.670030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fad97-99e2-4cc6-8525-9900d9fee37e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:13.670707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fad97-99e2-4cc6-8525-9900d9fee37e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:13.672327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fad97-99e2-4cc6-8525-9900d9fee37e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:13.685606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:13.714522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:13.754023 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fad97-99e2-4cc6-8525-9900d9fee37e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:13.755909 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fad97-99e2-4cc6-8525-9900d9fee37e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:13.829137 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26060517-e131-4cde-804e-6834aa81f6ee None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:13.862916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 152/291] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:13 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 15:00:13.868171 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fad97-99e2-4cc6-8525-9900d9fee37e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:13.930588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccd774-18db-42ee-890f-dc2cc2bc7141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:13.931318 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccd774-18db-42ee-890f-dc2cc2bc7141 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:13.931607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccd774-18db-42ee-890f-dc2cc2bc7141 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:13.931891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccd774-18db-42ee-890f-dc2cc2bc7141 None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f/users/0bbb19525e8f45e6bc60406c4972e45e/roles/471fd358281343cba97902b70b416f32` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:13.935018 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:13.943987 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccd774-18db-42ee-890f-dc2cc2bc7141 None admin] RBAC: Authorizing `identity:create_grant(project_id=ebb7ad89c9734ecbbd41c0c2de27069f, user_id=0bbb19525e8f45e6bc60406c4972e45e, role_id=471fd358281343cba97902b70b416f32)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:13.946453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccd774-18db-42ee-890f-dc2cc2bc7141 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:13.982132 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 140/292] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:00:13 2022] PUT /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f/users/0bbb19525e8f45e6bc60406c4972e45e/roles/471fd358281343cba97902b70b416f32 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:13.986970 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fad97-99e2-4cc6-8525-9900d9fee37e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:14.062100 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f3cce-d795-43a5-af5d-c0223a85be7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:14.062746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f3cce-d795-43a5-af5d-c0223a85be7d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:14.063104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f3cce-d795-43a5-af5d-c0223a85be7d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:14.063390 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f3cce-d795-43a5-af5d-c0223a85be7d None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:14.067739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f3cce-d795-43a5-af5d-c0223a85be7d None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:14.069322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f3cce-d795-43a5-af5d-c0223a85be7d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:14.072041 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 153/293] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:13 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:14.078298 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccd774-18db-42ee-890f-dc2cc2bc7141 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:14.228379 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccd774-18db-42ee-890f-dc2cc2bc7141 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:14.252636 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:14.267670 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1ffed-857b-45a6-8010-7a95c1769345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:14.268473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1ffed-857b-45a6-8010-7a95c1769345 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:14.268813 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1ffed-857b-45a6-8010-7a95c1769345 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:14.269153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1ffed-857b-45a6-8010-7a95c1769345 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:14.269961 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1ffed-857b-45a6-8010-7a95c1769345 None admin] RBAC: Adding query filter params (role.id=d1f6973ce11a4644b649b38f8b2df173, scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f, user.id=d30378f810a1497f94191b23bd742912) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:14.270279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1ffed-857b-45a6-8010-7a95c1769345 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:14.271858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1ffed-857b-45a6-8010-7a95c1769345 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:14.279209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 141/294] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:14 2022] GET /identity/v3/role_assignments?user.id=d30378f810a1497f94191b23bd742912&scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f&role.id=d1f6973ce11a4644b649b38f8b2df173 => generated 610 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:14.704746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe11f2c-224b-4e0b-a914-2902f03c4374 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:14.705304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe11f2c-224b-4e0b-a914-2902f03c4374 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:14.705597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe11f2c-224b-4e0b-a914-2902f03c4374 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:14.707536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 154/295] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:14 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:14.717179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d81426-4412-435d-9d58-1f6a2635c5db None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:14.717179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d81426-4412-435d-9d58-1f6a2635c5db None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:14.717179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d81426-4412-435d-9d58-1f6a2635c5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:14.735461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c9d81426-4412-435d-9d58-1f6a2635c5db None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:14.736985 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c9d81426-4412-435d-9d58-1f6a2635c5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:14.811630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 142/296] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:14.834698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe11f2c-224b-4e0b-a914-2902f03c4374 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:14.946588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe11f2c-224b-4e0b-a914-2902f03c4374 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:14.955942 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:14.985098 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03431a39-be8a-4120-b4fa-b01702ee640c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:14.986146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03431a39-be8a-4120-b4fa-b01702ee640c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:14.986629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03431a39-be8a-4120-b4fa-b01702ee640c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:14.987116 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03431a39-be8a-4120-b4fa-b01702ee640c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:15.000110 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03431a39-be8a-4120-b4fa-b01702ee640c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-03431a39-be8a-4120-b4fa-b01702ee640c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:15.005742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:15.006709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:15.007564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:15.007564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:15.007564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:15.007564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:15.007564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:15.007564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:15.007564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:15.007564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:15.007564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:15.007564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:15.007564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:15.007564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:15.010304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 155/297] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:15.016376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d81426-4412-435d-9d58-1f6a2635c5db None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:15.123574 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d81426-4412-435d-9d58-1f6a2635c5db None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:15.143348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:15.144671 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06869f68-9f27-44f6-9de5-d9450a5e2164 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:15.145017 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06869f68-9f27-44f6-9de5-d9450a5e2164 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:15.145531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06869f68-9f27-44f6-9de5-d9450a5e2164 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:15.147235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 156/298] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:00:15.153710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca821a-dec4-447a-8b95-82f2247d36c8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:15.154546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca821a-dec4-447a-8b95-82f2247d36c8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:15.155059 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca821a-dec4-447a-8b95-82f2247d36c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:15.167106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9feaaf-6b8d-4a5c-92ac-4a73880daece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:15.167727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9feaaf-6b8d-4a5c-92ac-4a73880daece None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:15.168007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9feaaf-6b8d-4a5c-92ac-4a73880daece None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:15.168279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9feaaf-6b8d-4a5c-92ac-4a73880daece None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:15.168998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9feaaf-6b8d-4a5c-92ac-4a73880daece None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:15.169299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9feaaf-6b8d-4a5c-92ac-4a73880daece None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:15.171231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9feaaf-6b8d-4a5c-92ac-4a73880daece None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:15.176376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-6eca821a-dec4-447a-8b95-82f2247d36c8 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:15.177927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6eca821a-dec4-447a-8b95-82f2247d36c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:15.183044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 143/299] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:15 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:15.189010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9feaaf-6b8d-4a5c-92ac-4a73880daece None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:15.262769 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 157/300] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:00:15.281256 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca821a-dec4-447a-8b95-82f2247d36c8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:15.324356 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9feaaf-6b8d-4a5c-92ac-4a73880daece None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:15.379274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95eddfb-ec65-42ad-a1ca-04bfdd0fa358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:15.380243 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eddfb-ec65-42ad-a1ca-04bfdd0fa358 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:15.380562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eddfb-ec65-42ad-a1ca-04bfdd0fa358 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:15.380850 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eddfb-ec65-42ad-a1ca-04bfdd0fa358 None admin] PATH_INFO: `/v3/users/00aad4fb761745499a6673f7b2bb88dd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:15.382807 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95eddfb-ec65-42ad-a1ca-04bfdd0fa358 None admin] RBAC: Authorizing `identity:get_user(user_id=00aad4fb761745499a6673f7b2bb88dd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:15.385461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95eddfb-ec65-42ad-a1ca-04bfdd0fa358 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:15.387271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:15.390758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 144/301] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:15 2022] GET /identity/v3/users/00aad4fb761745499a6673f7b2bb88dd => generated 294 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:15.392470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe479d-9e7b-48d5-b52d-0d6aeedafc33 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:15.392818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe479d-9e7b-48d5-b52d-0d6aeedafc33 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:15.393105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe479d-9e7b-48d5-b52d-0d6aeedafc33 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:15.395117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 145/302] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:15.395861 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe479d-9e7b-48d5-b52d-0d6aeedafc33 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:15.401932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca821a-dec4-447a-8b95-82f2247d36c8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:15.452381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:15.456924 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9be70-3485-4f5c-b649-00fd16c5092a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:15.457764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9be70-3485-4f5c-b649-00fd16c5092a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:15.458258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9be70-3485-4f5c-b649-00fd16c5092a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:15.458779 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9be70-3485-4f5c-b649-00fd16c5092a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:15.461121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9be70-3485-4f5c-b649-00fd16c5092a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:15.464016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9be70-3485-4f5c-b649-00fd16c5092a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:15.471063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 158/303] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:15 2022] GET /identity/v3/domains/default => generated 200 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:15.471063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d786f-ba5c-4906-8f89-fcb1a4567a9c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:15.471063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d786f-ba5c-4906-8f89-fcb1a4567a9c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:15.471063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d786f-ba5c-4906-8f89-fcb1a4567a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:15.497501 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-737d786f-ba5c-4906-8f89-fcb1a4567a9c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:15.499526 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-737d786f-ba5c-4906-8f89-fcb1a4567a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:15.527402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe479d-9e7b-48d5-b52d-0d6aeedafc33 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:15.572747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc2a9d-31c4-4914-97b2-374806e06ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:15.573541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc2a9d-31c4-4914-97b2-374806e06ce3 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:15.573676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc2a9d-31c4-4914-97b2-374806e06ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:15.573951 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc2a9d-31c4-4914-97b2-374806e06ce3 None admin] PATH_INFO: `/v3/system/users/00aad4fb761745499a6673f7b2bb88dd/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:15.575844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc2a9d-31c4-4914-97b2-374806e06ce3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=00aad4fb761745499a6673f7b2bb88dd, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:15.576838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc2a9d-31c4-4914-97b2-374806e06ce3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:15.587622 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 146/304] 10.222.0.100 () {58 vars in 1339 bytes} [Mon Aug 1 15:00:15 2022] PUT /identity/v3/system/users/00aad4fb761745499a6673f7b2bb88dd/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:15.589123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc2a9d-31c4-4914-97b2-374806e06ce3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:15.601497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 159/305] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:15.625913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d786f-ba5c-4906-8f89-fcb1a4567a9c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:15.739807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:15.748592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d786f-ba5c-4906-8f89-fcb1a4567a9c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:15.749565 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc2a9d-31c4-4914-97b2-374806e06ce3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:15.765444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:15.802647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b81a7-7cf5-4ac8-be35-78661a4bf9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:15.803308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b81a7-7cf5-4ac8-be35-78661a4bf9e9 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:15.803590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b81a7-7cf5-4ac8-be35-78661a4bf9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:15.803967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b81a7-7cf5-4ac8-be35-78661a4bf9e9 None admin] PATH_INFO: `/v3/groups` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:15.804423 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25e99c-6efe-40c7-a105-3d10e8c75a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:15.804678 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b81a7-7cf5-4ac8-be35-78661a4bf9e9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:15.805298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25e99c-6efe-40c7-a105-3d10e8c75a78 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:15.805649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25e99c-6efe-40c7-a105-3d10e8c75a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:15.805994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25e99c-6efe-40c7-a105-3d10e8c75a78 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:15.806149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b81a7-7cf5-4ac8-be35-78661a4bf9e9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:15.822031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25e99c-6efe-40c7-a105-3d10e8c75a78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:15.826023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 147/306] 10.222.0.100 () {60 vars in 1139 bytes} [Mon Aug 1 15:00:15 2022] POST /identity/v3/groups => generated 230 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-da25e99c-6efe-40c7-a105-3d10e8c75a78 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:15.828091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:15.829226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:15.830636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:15.830636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:15.830636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:15.830636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:15.830636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:15.830636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:15.830636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:15.830636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:15.830636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:15.830636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:15.830636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:00:15.830636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:15.830636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 160/307] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:15.842815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b81a7-7cf5-4ac8-be35-78661a4bf9e9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:15.973664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b81a7-7cf5-4ac8-be35-78661a4bf9e9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:16.014791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a892d-ef77-42a1-9acd-3d660868d104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:16.015356 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a892d-ef77-42a1-9acd-3d660868d104 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:16.015638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a892d-ef77-42a1-9acd-3d660868d104 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:16.015911 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a892d-ef77-42a1-9acd-3d660868d104 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:16.016667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a892d-ef77-42a1-9acd-3d660868d104 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:16.016974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a892d-ef77-42a1-9acd-3d660868d104 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:16.018369 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a892d-ef77-42a1-9acd-3d660868d104 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:16.028034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 148/308] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:15 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:16.033460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25e99c-6efe-40c7-a105-3d10e8c75a78 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:16.159399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25e99c-6efe-40c7-a105-3d10e8c75a78 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:16.217256 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff47e96-8f54-4c06-a6d7-a7c3bbfecc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:16.218887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47e96-8f54-4c06-a6d7-a7c3bbfecc1e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:16.219532 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47e96-8f54-4c06-a6d7-a7c3bbfecc1e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:16.220020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff47e96-8f54-4c06-a6d7-a7c3bbfecc1e None admin] PATH_INFO: `/v3/users/d0cfe09ac8dc4244b9f1117752c03e5a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:16.245010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff47e96-8f54-4c06-a6d7-a7c3bbfecc1e None admin] RBAC: Authorizing `identity:get_user(user_id=d0cfe09ac8dc4244b9f1117752c03e5a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:16.248005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff47e96-8f54-4c06-a6d7-a7c3bbfecc1e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:16.251837 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 161/309] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:16 2022] GET /identity/v3/users/d0cfe09ac8dc4244b9f1117752c03e5a => generated 294 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:16.258095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a892d-ef77-42a1-9acd-3d660868d104 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:16.298381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:16.384277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a892d-ef77-42a1-9acd-3d660868d104 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:16.385565 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6ac4e-e425-4e22-a92d-7ab0304f4b45 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:16.385931 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6ac4e-e425-4e22-a92d-7ab0304f4b45 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:16.386313 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6ac4e-e425-4e22-a92d-7ab0304f4b45 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:16.387895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 162/310] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:00:16.393354 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea575a26-2af5-4f98-a791-fefd9fb5e911 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:16.393629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea575a26-2af5-4f98-a791-fefd9fb5e911 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:16.393907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea575a26-2af5-4f98-a791-fefd9fb5e911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:16.416801 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ea575a26-2af5-4f98-a791-fefd9fb5e911 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:16.419208 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ea575a26-2af5-4f98-a791-fefd9fb5e911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:16.428353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b094c65-529d-4484-87fe-b143abb8dd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:16.429953 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b094c65-529d-4484-87fe-b143abb8dd35 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:16.430415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b094c65-529d-4484-87fe-b143abb8dd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:16.430741 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b094c65-529d-4484-87fe-b143abb8dd35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:16.431888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:16.432099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b094c65-529d-4484-87fe-b143abb8dd35 None admin] RBAC: Adding query filter params (role.id=471fd358281343cba97902b70b416f32, user.id=d0cfe09ac8dc4244b9f1117752c03e5a, scope.system=all) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:16.432862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b094c65-529d-4484-87fe-b143abb8dd35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:16.435534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b094c65-529d-4484-87fe-b143abb8dd35 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:16.441894 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 149/311] 10.222.0.100 () {56 vars in 1313 bytes} [Mon Aug 1 15:00:16 2022] GET /identity/v3/role_assignments?user.id=d0cfe09ac8dc4244b9f1117752c03e5a&scope.system=all&role.id=471fd358281343cba97902b70b416f32 => generated 227 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:16.529796 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 163/312] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:16 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:00:16.569926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b094c65-529d-4484-87fe-b143abb8dd35 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:16.683905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:16.690921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b094c65-529d-4484-87fe-b143abb8dd35 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:16.731083 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603bf82-bfbd-4a7a-bca9-69cf6e5b8499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:16.731823 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603bf82-bfbd-4a7a-bca9-69cf6e5b8499 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:16.732105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603bf82-bfbd-4a7a-bca9-69cf6e5b8499 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:16.732426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603bf82-bfbd-4a7a-bca9-69cf6e5b8499 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:16.744351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0603bf82-bfbd-4a7a-bca9-69cf6e5b8499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0603bf82-bfbd-4a7a-bca9-69cf6e5b8499 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:16.749913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:16.751323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:16.752123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:16.752123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:16.752123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:16.752123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:16.752123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:16.752123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:16.752123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:16.752123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:16.752123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:16.752123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:16.752123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:00:16.752123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:16.752123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 150/313] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:16.760024 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea575a26-2af5-4f98-a791-fefd9fb5e911 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:16.897420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea575a26-2af5-4f98-a791-fefd9fb5e911 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:16.934381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da004bf-1769-4d3b-916e-65d267347ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:16.935034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da004bf-1769-4d3b-916e-65d267347ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:16.935314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da004bf-1769-4d3b-916e-65d267347ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:16.935587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da004bf-1769-4d3b-916e-65d267347ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:16.936307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da004bf-1769-4d3b-916e-65d267347ae2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:16.936609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da004bf-1769-4d3b-916e-65d267347ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:16.938166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da004bf-1769-4d3b-916e-65d267347ae2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:16.947882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 164/314] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:16 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:16.956269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603bf82-bfbd-4a7a-bca9-69cf6e5b8499 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:17.120899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603bf82-bfbd-4a7a-bca9-69cf6e5b8499 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:17.158066 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414ec15-37a3-4dad-b279-c1caae5b688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:17.158638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414ec15-37a3-4dad-b279-c1caae5b688d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:17.158955 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414ec15-37a3-4dad-b279-c1caae5b688d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:17.159372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414ec15-37a3-4dad-b279-c1caae5b688d None admin] PATH_INFO: `/v3/users/0bbb19525e8f45e6bc60406c4972e45e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:17.160654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414ec15-37a3-4dad-b279-c1caae5b688d None admin] RBAC: Authorizing `identity:get_user(user_id=0bbb19525e8f45e6bc60406c4972e45e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:17.162241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414ec15-37a3-4dad-b279-c1caae5b688d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:17.164790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 151/315] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:16 2022] GET /identity/v3/users/0bbb19525e8f45e6bc60406c4972e45e => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:17.171866 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da004bf-1769-4d3b-916e-65d267347ae2 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:17.175831 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:17.296051 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da004bf-1769-4d3b-916e-65d267347ae2 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:17.339317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd1228-3a3c-4d46-b436-c90ccc091064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:17.340046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd1228-3a3c-4d46-b436-c90ccc091064 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:17.340328 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd1228-3a3c-4d46-b436-c90ccc091064 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:17.340604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd1228-3a3c-4d46-b436-c90ccc091064 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:17.412928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd1228-3a3c-4d46-b436-c90ccc091064 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:17.414954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd1228-3a3c-4d46-b436-c90ccc091064 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:17.423607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:17.521362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 165/316] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:17 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:17.530414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414ec15-37a3-4dad-b279-c1caae5b688d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:17.577540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:17.718647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414ec15-37a3-4dad-b279-c1caae5b688d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:17.760875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae3dbe-a8be-4f96-b610-baeb67171e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:17.762342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae3dbe-a8be-4f96-b610-baeb67171e3b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:17.762895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae3dbe-a8be-4f96-b610-baeb67171e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:17.763379 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae3dbe-a8be-4f96-b610-baeb67171e3b None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:17.764805 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae3dbe-a8be-4f96-b610-baeb67171e3b None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:17.766569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae3dbe-a8be-4f96-b610-baeb67171e3b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:17.769045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 152/317] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:17 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:17.776241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd1228-3a3c-4d46-b436-c90ccc091064 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:17.814783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47523a5b-4b90-40a3-a0b8-79038ca5df83 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:17.815088 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47523a5b-4b90-40a3-a0b8-79038ca5df83 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:17.815352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47523a5b-4b90-40a3-a0b8-79038ca5df83 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:17.816986 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 153/318] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:00:17.822954 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05f7b8-98d2-434f-87c8-84fe0631d001 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:17.823256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05f7b8-98d2-434f-87c8-84fe0631d001 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:17.823524 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05f7b8-98d2-434f-87c8-84fe0631d001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:17.842922 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0c05f7b8-98d2-434f-87c8-84fe0631d001 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:17.844135 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0c05f7b8-98d2-434f-87c8-84fe0631d001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:17.915267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd1228-3a3c-4d46-b436-c90ccc091064 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:17.919277 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:17.940630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 154/319] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:17 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:00:17.965927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79c7a3-dc44-4dec-979c-205a351898ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:17.966565 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79c7a3-dc44-4dec-979c-205a351898ea None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:17.966887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79c7a3-dc44-4dec-979c-205a351898ea None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:17.967218 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79c7a3-dc44-4dec-979c-205a351898ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:17.968135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79c7a3-dc44-4dec-979c-205a351898ea None admin] RBAC: Adding query filter params (role.id=471fd358281343cba97902b70b416f32, scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f, user.id=0bbb19525e8f45e6bc60406c4972e45e) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:17.968715 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79c7a3-dc44-4dec-979c-205a351898ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:17.969034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05f7b8-98d2-434f-87c8-84fe0631d001 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:17.970649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79c7a3-dc44-4dec-979c-205a351898ea None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:17.979206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 166/320] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:17 2022] GET /identity/v3/role_assignments?user.id=0bbb19525e8f45e6bc60406c4972e45e&scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f&role.id=471fd358281343cba97902b70b416f32 => generated 610 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:18.005300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:18.118926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05f7b8-98d2-434f-87c8-84fe0631d001 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:18.164856 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f67ff6-0c8c-4e32-a7ae-6987a5244a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:18.165917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f67ff6-0c8c-4e32-a7ae-6987a5244a7a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:18.166406 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f67ff6-0c8c-4e32-a7ae-6987a5244a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:18.166957 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f67ff6-0c8c-4e32-a7ae-6987a5244a7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:18.181538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f67ff6-0c8c-4e32-a7ae-6987a5244a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f6f67ff6-0c8c-4e32-a7ae-6987a5244a7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:18.187736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:18.188676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:18.189492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:18.189492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:18.189492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:18.189492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:18.189492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:18.189492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:18.189492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:18.189492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:18.189492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:18.189492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:18.189492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:18.189492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:18.192455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 155/321] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:18.199832 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79c7a3-dc44-4dec-979c-205a351898ea None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:18.346179 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79c7a3-dc44-4dec-979c-205a351898ea None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:18.391847 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7feb59a-a646-41d8-8b54-40b36724c56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:18.392874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7feb59a-a646-41d8-8b54-40b36724c56a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:18.393317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7feb59a-a646-41d8-8b54-40b36724c56a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:18.393662 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7feb59a-a646-41d8-8b54-40b36724c56a None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:18.394449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7feb59a-a646-41d8-8b54-40b36724c56a None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:18.394839 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7feb59a-a646-41d8-8b54-40b36724c56a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:18.396202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7feb59a-a646-41d8-8b54-40b36724c56a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:18.407111 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70f40-59ee-44d2-b4d6-39b8f829ff34 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:18.407470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70f40-59ee-44d2-b4d6-39b8f829ff34 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:18.407641 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 167/322] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:18 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:18.409096 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70f40-59ee-44d2-b4d6-39b8f829ff34 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:18.410401 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 156/323] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:18 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:00:18.413938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7feb59a-a646-41d8-8b54-40b36724c56a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:18.417028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09fcce-9495-43e5-8e07-1fcdacb84e4b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:18.417410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09fcce-9495-43e5-8e07-1fcdacb84e4b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:18.417683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09fcce-9495-43e5-8e07-1fcdacb84e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:18.436841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3a09fcce-9495-43e5-8e07-1fcdacb84e4b None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:18.438179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3a09fcce-9495-43e5-8e07-1fcdacb84e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:18.468867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:18.568801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 157/324] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:00:18.575785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7feb59a-a646-41d8-8b54-40b36724c56a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:18.597232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09fcce-9495-43e5-8e07-1fcdacb84e4b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:18.615982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c956da7-be4e-42a3-90af-0ef77093006a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:18.616723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c956da7-be4e-42a3-90af-0ef77093006a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:18.617593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:18.617789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c956da7-be4e-42a3-90af-0ef77093006a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:18.617925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c956da7-be4e-42a3-90af-0ef77093006a None admin] PATH_INFO: `/v3/users/00aad4fb761745499a6673f7b2bb88dd` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:18.619379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c956da7-be4e-42a3-90af-0ef77093006a None admin] RBAC: Authorizing `identity:get_user(user_id=00aad4fb761745499a6673f7b2bb88dd)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:18.621072 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c956da7-be4e-42a3-90af-0ef77093006a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:18.622954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 168/325] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:18 2022] GET /identity/v3/users/00aad4fb761745499a6673f7b2bb88dd => generated 294 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:18.629429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c956da7-be4e-42a3-90af-0ef77093006a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:18.734936 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09fcce-9495-43e5-8e07-1fcdacb84e4b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:18.763483 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:18.768560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c956da7-be4e-42a3-90af-0ef77093006a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:18.774237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278d4f9-1fa7-4630-8cad-9861fa91b47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:18.774935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278d4f9-1fa7-4630-8cad-9861fa91b47f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:18.775203 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278d4f9-1fa7-4630-8cad-9861fa91b47f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:18.775474 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278d4f9-1fa7-4630-8cad-9861fa91b47f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:18.776650 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278d4f9-1fa7-4630-8cad-9861fa91b47f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:18.778108 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6278d4f9-1fa7-4630-8cad-9861fa91b47f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:18.779778 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 158/326] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:18 2022] GET /identity/v3/domains/default => generated 200 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:18.784664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa238f0-bef5-4cbc-a07e-2843311ac6da None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:18.784943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa238f0-bef5-4cbc-a07e-2843311ac6da None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:18.785212 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa238f0-bef5-4cbc-a07e-2843311ac6da None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:18.786552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 159/327] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:00:18.787334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa238f0-bef5-4cbc-a07e-2843311ac6da None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:18.811457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a9ac8-839d-4e08-96aa-38b0d584972c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:18.812095 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a9ac8-839d-4e08-96aa-38b0d584972c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:18.812381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a9ac8-839d-4e08-96aa-38b0d584972c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:18.812654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a9ac8-839d-4e08-96aa-38b0d584972c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:18.813452 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a9ac8-839d-4e08-96aa-38b0d584972c None admin] RBAC: Adding query filter params (role.id=d1f6973ce11a4644b649b38f8b2df173, user.id=00aad4fb761745499a6673f7b2bb88dd, scope.system=all) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:18.813766 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a9ac8-839d-4e08-96aa-38b0d584972c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:18.815252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a9ac8-839d-4e08-96aa-38b0d584972c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:18.828468 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 169/328] 10.222.0.100 () {56 vars in 1313 bytes} [Mon Aug 1 15:00:18 2022] GET /identity/v3/role_assignments?user.id=00aad4fb761745499a6673f7b2bb88dd&scope.system=all&role.id=d1f6973ce11a4644b649b38f8b2df173 => generated 512 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:18.829760 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1ef14-c6d9-4bb5-b561-bd65ce418548 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:18.829987 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1ef14-c6d9-4bb5-b561-bd65ce418548 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:18.830270 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1ef14-c6d9-4bb5-b561-bd65ce418548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:18.857229 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-60d1ef14-c6d9-4bb5-b561-bd65ce418548 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:18.858697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-60d1ef14-c6d9-4bb5-b561-bd65ce418548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:18.882114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:18.907185 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa238f0-bef5-4cbc-a07e-2843311ac6da None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:18.950708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0a137-af5b-4378-afce-036176b0b540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:18.950923 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 170/329] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:18.951007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0a137-af5b-4378-afce-036176b0b540 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:18.951007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0a137-af5b-4378-afce-036176b0b540 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:18.951007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0a137-af5b-4378-afce-036176b0b540 None admin] PATH_INFO: `/v3/groups` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:18.951007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0a137-af5b-4378-afce-036176b0b540 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:18.951007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b0a137-af5b-4378-afce-036176b0b540 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:18.970722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 160/330] 10.222.0.100 () {60 vars in 1139 bytes} [Mon Aug 1 15:00:18 2022] POST /identity/v3/groups => generated 227 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:00:18.976797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1ef14-c6d9-4bb5-b561-bd65ce418548 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:19.110425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:19.130635 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1ef14-c6d9-4bb5-b561-bd65ce418548 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:19.172725 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c38d28-11ba-4986-bec2-a2003bea3d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:19.173374 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c38d28-11ba-4986-bec2-a2003bea3d41 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:19.173653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c38d28-11ba-4986-bec2-a2003bea3d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:19.173922 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c38d28-11ba-4986-bec2-a2003bea3d41 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:19.184682 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c38d28-11ba-4986-bec2-a2003bea3d41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-79c38d28-11ba-4986-bec2-a2003bea3d41 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:19.189805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:19.190786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:19.191597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:19.191597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:19.191597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:19.191597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:19.191597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:19.191597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:19.191597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:19.191597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:19.191597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:19.191597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:19.191597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:00:19.191597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:19.191597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 171/331] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:19.197305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0a137-af5b-4378-afce-036176b0b540 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:19.328494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0a137-af5b-4378-afce-036176b0b540 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:19.366534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ddcf4b-735a-4f2e-ba8e-e5fbf16ab86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:19.367155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddcf4b-735a-4f2e-ba8e-e5fbf16ab86c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:19.367444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddcf4b-735a-4f2e-ba8e-e5fbf16ab86c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:19.367724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddcf4b-735a-4f2e-ba8e-e5fbf16ab86c None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:19.368464 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ddcf4b-735a-4f2e-ba8e-e5fbf16ab86c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:19.368782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ddcf4b-735a-4f2e-ba8e-e5fbf16ab86c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:19.370183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ddcf4b-735a-4f2e-ba8e-e5fbf16ab86c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:19.379526 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 161/332] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:19 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:19.385849 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c38d28-11ba-4986-bec2-a2003bea3d41 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:19.536336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c38d28-11ba-4986-bec2-a2003bea3d41 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:19.598392 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625331d-f735-46b8-853c-5597ecf83956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:19.599092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625331d-f735-46b8-853c-5597ecf83956 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:19.599377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625331d-f735-46b8-853c-5597ecf83956 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:19.599651 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625331d-f735-46b8-853c-5597ecf83956 None admin] PATH_INFO: `/v3/users/d0cfe09ac8dc4244b9f1117752c03e5a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:19.601464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625331d-f735-46b8-853c-5597ecf83956 None admin] RBAC: Authorizing `identity:get_user(user_id=d0cfe09ac8dc4244b9f1117752c03e5a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:19.602979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625331d-f735-46b8-853c-5597ecf83956 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:19.604957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 172/333] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:19 2022] GET /identity/v3/users/d0cfe09ac8dc4244b9f1117752c03e5a => generated 294 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:19.611410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ddcf4b-735a-4f2e-ba8e-e5fbf16ab86c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:19.616742 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:19.768088 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ddcf4b-735a-4f2e-ba8e-e5fbf16ab86c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:19.822570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f3fd8-aad7-4c66-a943-4268b56bb38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:19.823349 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f3fd8-aad7-4c66-a943-4268b56bb38a None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:19.823537 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f3fd8-aad7-4c66-a943-4268b56bb38a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:19.823819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f3fd8-aad7-4c66-a943-4268b56bb38a None admin] PATH_INFO: `/v3/system/users/d0cfe09ac8dc4244b9f1117752c03e5a/roles/471fd358281343cba97902b70b416f32` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:19.825991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f3fd8-aad7-4c66-a943-4268b56bb38a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d0cfe09ac8dc4244b9f1117752c03e5a, role_id=471fd358281343cba97902b70b416f32)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:19.827547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f3fd8-aad7-4c66-a943-4268b56bb38a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:19.839818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 162/334] 10.222.0.100 () {58 vars in 1339 bytes} [Mon Aug 1 15:00:19 2022] PUT /identity/v3/system/users/d0cfe09ac8dc4244b9f1117752c03e5a/roles/471fd358281343cba97902b70b416f32 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:21.682646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f21441-5b49-48b6-a0a0-ea4566244d25 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:21.683508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f21441-5b49-48b6-a0a0-ea4566244d25 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:21.683508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f21441-5b49-48b6-a0a0-ea4566244d25 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:21.684801 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 173/335] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:00:21.689970 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20abfbb-cf8c-4b4f-ad79-bef55b69321b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:21.690692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20abfbb-cf8c-4b4f-ad79-bef55b69321b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:21.690692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20abfbb-cf8c-4b4f-ad79-bef55b69321b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:21.719960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a20abfbb-cf8c-4b4f-ad79-bef55b69321b None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:21.721255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a20abfbb-cf8c-4b4f-ad79-bef55b69321b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:21.799755 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:21.811061 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 163/336] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:21 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:21.847959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f21441-5b49-48b6-a0a0-ea4566244d25 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:21.887948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:22.009158 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f03a5-f1b4-4d9c-bf3c-e6d03975553e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:22.009484 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f03a5-f1b4-4d9c-bf3c-e6d03975553e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:22.009736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f03a5-f1b4-4d9c-bf3c-e6d03975553e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:22.011305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 164/337] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:22 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:00:22.020067 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d19557-8a68-40b1-87a7-21013a7b26f5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:22.020334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d19557-8a68-40b1-87a7-21013a7b26f5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:22.020596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d19557-8a68-40b1-87a7-21013a7b26f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:22.051635 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f21441-5b49-48b6-a0a0-ea4566244d25 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:22.063763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-79d19557-8a68-40b1-87a7-21013a7b26f5 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:22.064995 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-79d19557-8a68-40b1-87a7-21013a7b26f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:22.140798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a183bd-194c-4060-90f7-7848f2204975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:22.141467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a183bd-194c-4060-90f7-7848f2204975 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:22.141754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a183bd-194c-4060-90f7-7848f2204975 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:22.142029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a183bd-194c-4060-90f7-7848f2204975 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:22.180179 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a183bd-194c-4060-90f7-7848f2204975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-81a183bd-194c-4060-90f7-7848f2204975 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:22.185264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:22.186174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:22.187021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:22.187021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:22.187021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:22.187021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:22.187021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:22.187021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:22.187021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:22.187021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:22.187021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:22.187021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:22.187021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:22.187021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:22.187021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 174/338] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:00:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:22.193989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a183bd-194c-4060-90f7-7848f2204975 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:22.236402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 165/339] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:22 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:22.288760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d19557-8a68-40b1-87a7-21013a7b26f5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:22.353927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:22.425772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a183bd-194c-4060-90f7-7848f2204975 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:22.528482 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab6489-e3c1-4e84-b1f7-3a0e2196302e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:22.529125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab6489-e3c1-4e84-b1f7-3a0e2196302e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:22.529404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab6489-e3c1-4e84-b1f7-3a0e2196302e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:22.529689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab6489-e3c1-4e84-b1f7-3a0e2196302e None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:22.530389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab6489-e3c1-4e84-b1f7-3a0e2196302e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:22.530723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab6489-e3c1-4e84-b1f7-3a0e2196302e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:22.532264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab6489-e3c1-4e84-b1f7-3a0e2196302e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:22.553198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 175/340] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:22 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:22.564119 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab6489-e3c1-4e84-b1f7-3a0e2196302e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:22.577696 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d19557-8a68-40b1-87a7-21013a7b26f5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:22.645011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:22.676799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e5d5d-95c0-4abd-a051-5e14b26e32a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:22.677474 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e5d5d-95c0-4abd-a051-5e14b26e32a4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:22.677760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e5d5d-95c0-4abd-a051-5e14b26e32a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:22.678041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e5d5d-95c0-4abd-a051-5e14b26e32a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:22.711863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e5d5d-95c0-4abd-a051-5e14b26e32a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-480e5d5d-95c0-4abd-a051-5e14b26e32a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:22.717189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:22.718918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:22.720267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:22.720267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:22.720267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:22.720267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:22.720267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:22.720267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:22.720267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:22.720267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:22.720267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:22.720267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:22.720267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:22.720267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:22.720267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 166/341] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:22.727706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e5d5d-95c0-4abd-a051-5e14b26e32a4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:22.819071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab6489-e3c1-4e84-b1f7-3a0e2196302e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:22.922405 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a47d1-2873-42c6-aee9-35f2423f90ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:22.930690 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a47d1-2873-42c6-aee9-35f2423f90ae None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:22.930690 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a47d1-2873-42c6-aee9-35f2423f90ae None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:22.930690 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a47d1-2873-42c6-aee9-35f2423f90ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:22.940702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:22.964251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e5d5d-95c0-4abd-a051-5e14b26e32a4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:23.016413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe677a4-822a-4f3a-94db-1c3c37b8ff45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:23.017421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe677a4-822a-4f3a-94db-1c3c37b8ff45 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.017905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe677a4-822a-4f3a-94db-1c3c37b8ff45 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.018394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe677a4-822a-4f3a-94db-1c3c37b8ff45 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.019384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe677a4-822a-4f3a-94db-1c3c37b8ff45 None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:23.019891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe677a4-822a-4f3a-94db-1c3c37b8ff45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:23.021575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe677a4-822a-4f3a-94db-1c3c37b8ff45 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:23.031981 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 167/342] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:22 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:23.033675 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de145bed-9613-4241-b3a1-9e02d07d0ef9 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.034191 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de145bed-9613-4241-b3a1-9e02d07d0ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.034683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de145bed-9613-4241-b3a1-9e02d07d0ef9 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.036753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 168/343] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:23 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:23.041951 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879dbac-79c6-4b68-9cbb-3557685f7299 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.042555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879dbac-79c6-4b68-9cbb-3557685f7299 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.043091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879dbac-79c6-4b68-9cbb-3557685f7299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.064721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a47d1-2873-42c6-aee9-35f2423f90ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:23.066197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a47d1-2873-42c6-aee9-35f2423f90ae None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:23.085323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4879dbac-79c6-4b68-9cbb-3557685f7299 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:23.091820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4879dbac-79c6-4b68-9cbb-3557685f7299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:23.172307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:23.185159 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 176/344] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:22 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:23.186063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a47d1-2873-42c6-aee9-35f2423f90ae None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:23.202872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 169/345] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:23 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:23.204575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee8b6-e03e-4b64-a350-4b5ea22d691d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.205089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee8b6-e03e-4b64-a350-4b5ea22d691d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.205587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee8b6-e03e-4b64-a350-4b5ea22d691d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.207133 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 170/346] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:23.208055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ee8b6-e03e-4b64-a350-4b5ea22d691d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:23.223997 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:23.298996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a47d1-2873-42c6-aee9-35f2423f90ae None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:23.334566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ee8b6-e03e-4b64-a350-4b5ea22d691d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:23.340042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7388df4-8f6a-4ffc-a377-93fba7fc6a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:23.340679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7388df4-8f6a-4ffc-a377-93fba7fc6a89 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.340955 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7388df4-8f6a-4ffc-a377-93fba7fc6a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.341224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7388df4-8f6a-4ffc-a377-93fba7fc6a89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.375718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe7cc48-d480-423c-8954-fab476f945a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:23.376746 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7cc48-d480-423c-8954-fab476f945a3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.377706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7cc48-d480-423c-8954-fab476f945a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.378199 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe7cc48-d480-423c-8954-fab476f945a3 None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.379830 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe7cc48-d480-423c-8954-fab476f945a3 None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:23.381861 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe7cc48-d480-423c-8954-fab476f945a3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:23.383935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 171/347] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:23 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:23.385381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c4b25-86e9-4af5-8109-ed61ea2984c9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.385869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c4b25-86e9-4af5-8109-ed61ea2984c9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.386337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c4b25-86e9-4af5-8109-ed61ea2984c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.407581 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-315c4b25-86e9-4af5-8109-ed61ea2984c9 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:23.409182 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-315c4b25-86e9-4af5-8109-ed61ea2984c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:23.419067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7388df4-8f6a-4ffc-a377-93fba7fc6a89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:23.420641 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7388df4-8f6a-4ffc-a377-93fba7fc6a89 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:23.458788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:23.459941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:23.491220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 172/348] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:23 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:23.492639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c4b25-86e9-4af5-8109-ed61ea2984c9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:23.542810 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 177/349] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:23 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:23.543766 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7388df4-8f6a-4ffc-a377-93fba7fc6a89 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:23.648610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c4b25-86e9-4af5-8109-ed61ea2984c9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:23.679872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7388df4-8f6a-4ffc-a377-93fba7fc6a89 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:23.694220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9971d-6fcb-4e9e-a8dc-46e102a6422d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:23.694983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9971d-6fcb-4e9e-a8dc-46e102a6422d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.695267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9971d-6fcb-4e9e-a8dc-46e102a6422d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.695547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9971d-6fcb-4e9e-a8dc-46e102a6422d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.708048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9971d-6fcb-4e9e-a8dc-46e102a6422d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-20e9971d-6fcb-4e9e-a8dc-46e102a6422d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:23.714550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:23.715923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:23.716758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:23.716758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:23.716758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:23.716758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:23.716758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:23.716758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:23.716758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:23.716758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:23.716758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:23.716758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:23.716758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:23.716758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:23.716758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 173/350] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:23.716758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9971d-6fcb-4e9e-a8dc-46e102a6422d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:23.740515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140990f7-cd57-41e8-800b-de6ebcd23bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:23.742521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140990f7-cd57-41e8-800b-de6ebcd23bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.742704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140990f7-cd57-41e8-800b-de6ebcd23bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.743409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140990f7-cd57-41e8-800b-de6ebcd23bd0 None admin] PATH_INFO: `/v3/groups/67c25d20e3e34c0b9d74a852083437c0` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.750751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:23.753781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140990f7-cd57-41e8-800b-de6ebcd23bd0 None admin] RBAC: Authorizing `identity:get_group(group_id=67c25d20e3e34c0b9d74a852083437c0)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:23.755550 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140990f7-cd57-41e8-800b-de6ebcd23bd0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:23.757646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 178/351] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:23 2022] GET /identity/v3/groups/67c25d20e3e34c0b9d74a852083437c0 => generated 227 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:23.759112 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915540b-97be-4da7-8cdc-fa2fb3fbc886 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.759845 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915540b-97be-4da7-8cdc-fa2fb3fbc886 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.760522 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915540b-97be-4da7-8cdc-fa2fb3fbc886 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.762487 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 179/352] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:23 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:23.763280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915540b-97be-4da7-8cdc-fa2fb3fbc886 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:23.781480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:23.864350 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9971d-6fcb-4e9e-a8dc-46e102a6422d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:23.909381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4505c-2492-4216-8260-30ef85464d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:23.910028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915540b-97be-4da7-8cdc-fa2fb3fbc886 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:23.911859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4505c-2492-4216-8260-30ef85464d3f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.911859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4505c-2492-4216-8260-30ef85464d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.911859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4505c-2492-4216-8260-30ef85464d3f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.928296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4505c-2492-4216-8260-30ef85464d3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-17f4505c-2492-4216-8260-30ef85464d3f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:23.934462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:23.935445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:23.936260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:23.936260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:23.936260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:23.936260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:23.936260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:23.936260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:23.936260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:23.936260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:23.936260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:23.936260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:23.936260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 15:00:23.936260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:23.936260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 174/353] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:23.936973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4505c-2492-4216-8260-30ef85464d3f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:23.966823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28101b3d-e636-444b-8127-ae94ac042b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:23.967709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28101b3d-e636-444b-8127-ae94ac042b7a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:23.968301 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28101b3d-e636-444b-8127-ae94ac042b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:23.968759 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28101b3d-e636-444b-8127-ae94ac042b7a None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:23.970633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28101b3d-e636-444b-8127-ae94ac042b7a None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:23.972159 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28101b3d-e636-444b-8127-ae94ac042b7a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:23.974036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 180/354] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:23 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:23.974651 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28101b3d-e636-444b-8127-ae94ac042b7a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:24.070283 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:24.093049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:24.101767 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4505c-2492-4216-8260-30ef85464d3f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:24.126248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28101b3d-e636-444b-8127-ae94ac042b7a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:24.163049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163f026-456b-4bb1-b22e-367f305506be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:24.164034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163f026-456b-4bb1-b22e-367f305506be None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:24.164496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163f026-456b-4bb1-b22e-367f305506be None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:24.164966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163f026-456b-4bb1-b22e-367f305506be None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:24.165783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163f026-456b-4bb1-b22e-367f305506be None admin] RBAC: Adding query filter params (name=member) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:24.166124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163f026-456b-4bb1-b22e-367f305506be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:24.167564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163f026-456b-4bb1-b22e-367f305506be None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:24.170224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc510ce1-44eb-41ab-8028-bb13b759291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:24.171889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc510ce1-44eb-41ab-8028-bb13b759291b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:24.172853 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc510ce1-44eb-41ab-8028-bb13b759291b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:24.172853 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc510ce1-44eb-41ab-8028-bb13b759291b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:24.174544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc510ce1-44eb-41ab-8028-bb13b759291b None admin] RBAC: Adding query filter params (group.id=67c25d20e3e34c0b9d74a852083437c0, role.id=57221d717a454c7fae4f2df1de9dbe5b, scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:24.174544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc510ce1-44eb-41ab-8028-bb13b759291b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:24.177235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc510ce1-44eb-41ab-8028-bb13b759291b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:24.180354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 175/355] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:23 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:24.182445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42d8fd-c7f9-462f-aae7-a6e4a5fb190e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:24.183172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42d8fd-c7f9-462f-aae7-a6e4a5fb190e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:24.183643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42d8fd-c7f9-462f-aae7-a6e4a5fb190e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:24.191042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 181/356] 10.222.0.100 () {56 vars in 1381 bytes} [Mon Aug 1 15:00:23 2022] GET /identity/v3/role_assignments?group.id=67c25d20e3e34c0b9d74a852083437c0&scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80&role.id=57221d717a454c7fae4f2df1de9dbe5b => generated 261 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:24.192282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc510ce1-44eb-41ab-8028-bb13b759291b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:24.210188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ac42d8fd-c7f9-462f-aae7-a6e4a5fb190e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:24.212399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ac42d8fd-c7f9-462f-aae7-a6e4a5fb190e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:24.314478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 176/357] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:24 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:24.315010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42d8fd-c7f9-462f-aae7-a6e4a5fb190e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:24.369022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc510ce1-44eb-41ab-8028-bb13b759291b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:24.418484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:24.427612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a85eb6-d436-423b-865b-4ac071bdd5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:24.427612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a85eb6-d436-423b-865b-4ac071bdd5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:24.427612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a85eb6-d436-423b-865b-4ac071bdd5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:24.427612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a85eb6-d436-423b-865b-4ac071bdd5a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:24.428005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a85eb6-d436-423b-865b-4ac071bdd5a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:24.429198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a85eb6-d436-423b-865b-4ac071bdd5a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:24.430289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a85eb6-d436-423b-865b-4ac071bdd5a3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:24.441307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 182/358] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:24 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:24.441307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066e74c-083c-4965-953c-f685261da4fb None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:24.441307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066e74c-083c-4965-953c-f685261da4fb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:24.441307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066e74c-083c-4965-953c-f685261da4fb None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:24.442454 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 183/359] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:24.445575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066e74c-083c-4965-953c-f685261da4fb None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:24.458285 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:24.480638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42d8fd-c7f9-462f-aae7-a6e4a5fb190e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:24.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806db98c-5a45-4ddc-9db8-212e5d3186df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:24.529966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806db98c-5a45-4ddc-9db8-212e5d3186df None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:24.530252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806db98c-5a45-4ddc-9db8-212e5d3186df None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:24.530534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806db98c-5a45-4ddc-9db8-212e5d3186df None admin] PATH_INFO: `/v3/groups/67c25d20e3e34c0b9d74a852083437c0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:24.540274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806db98c-5a45-4ddc-9db8-212e5d3186df None admin] RBAC: Authorizing `identity:get_group(group_id=67c25d20e3e34c0b9d74a852083437c0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:24.542084 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806db98c-5a45-4ddc-9db8-212e5d3186df None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:24.546899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 177/360] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:24 2022] GET /identity/v3/groups/67c25d20e3e34c0b9d74a852083437c0 => generated 227 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:24.550111 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806db98c-5a45-4ddc-9db8-212e5d3186df None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:24.600520 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066e74c-083c-4965-953c-f685261da4fb None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:24.644173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c363bd-76a2-4264-9a1a-e846d6f68b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:24.645152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c363bd-76a2-4264-9a1a-e846d6f68b82 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:24.645416 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c363bd-76a2-4264-9a1a-e846d6f68b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:24.645677 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c363bd-76a2-4264-9a1a-e846d6f68b82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:24.720016 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806db98c-5a45-4ddc-9db8-212e5d3186df None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:24.746000 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:24.763958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c363bd-76a2-4264-9a1a-e846d6f68b82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:24.765602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c363bd-76a2-4264-9a1a-e846d6f68b82 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:24.775511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748f8da-a2b9-4668-8be9-4f52874f08f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:24.776609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748f8da-a2b9-4668-8be9-4f52874f08f9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:24.777246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748f8da-a2b9-4668-8be9-4f52874f08f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:24.777925 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5748f8da-a2b9-4668-8be9-4f52874f08f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:24.795709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5748f8da-a2b9-4668-8be9-4f52874f08f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5748f8da-a2b9-4668-8be9-4f52874f08f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:24.800802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:24.801791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:24.802689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:24.802689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:24.802689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:24.802689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:24.802689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:24.802689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:24.802689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:24.802689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:24.802689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:24.802689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:24.802689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:24.802689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:24.802689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 178/361] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:00:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:24.803460 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748f8da-a2b9-4668-8be9-4f52874f08f9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:24.836057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:24.913071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 184/362] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:24 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:24.916038 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d6402-d362-4392-bd58-6b3ddfd130c0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:24.916631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d6402-d362-4392-bd58-6b3ddfd130c0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:24.917068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d6402-d362-4392-bd58-6b3ddfd130c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:24.936882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a16d6402-d362-4392-bd58-6b3ddfd130c0 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:24.938120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a16d6402-d362-4392-bd58-6b3ddfd130c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:24.952226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5748f8da-a2b9-4668-8be9-4f52874f08f9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:25.005495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fbc71-ac70-4a05-8d04-8034046227f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:25.005495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fbc71-ac70-4a05-8d04-8034046227f7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:25.005814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fbc71-ac70-4a05-8d04-8034046227f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:25.005917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fbc71-ac70-4a05-8d04-8034046227f7 None admin] PATH_INFO: `/v3/users/d0cfe09ac8dc4244b9f1117752c03e5a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:25.007239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fbc71-ac70-4a05-8d04-8034046227f7 None admin] RBAC: Authorizing `identity:get_user(user_id=d0cfe09ac8dc4244b9f1117752c03e5a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:25.009496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fbc71-ac70-4a05-8d04-8034046227f7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:25.011662 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 179/363] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:00:24 2022] GET /identity/v3/users/d0cfe09ac8dc4244b9f1117752c03e5a => generated 294 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:25.013363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fbc71-ac70-4a05-8d04-8034046227f7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:25.040153 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 185/364] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:24 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:25.041646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d6402-d362-4392-bd58-6b3ddfd130c0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:25.079862 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:25.122814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:25.143775 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fbc71-ac70-4a05-8d04-8034046227f7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:25.184242 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d6402-d362-4392-bd58-6b3ddfd130c0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:25.204312 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c189b3-1d53-4a8f-82f6-dc443d52a7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:25.204938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c189b3-1d53-4a8f-82f6-dc443d52a7ba None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:25.205228 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c189b3-1d53-4a8f-82f6-dc443d52a7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:25.205503 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c189b3-1d53-4a8f-82f6-dc443d52a7ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:25.206322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c189b3-1d53-4a8f-82f6-dc443d52a7ba None admin] RBAC: Adding query filter params (group.id=67c25d20e3e34c0b9d74a852083437c0, role.id=d1f6973ce11a4644b649b38f8b2df173, scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:25.206842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c189b3-1d53-4a8f-82f6-dc443d52a7ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:25.208304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c189b3-1d53-4a8f-82f6-dc443d52a7ba None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:25.217123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 180/365] 10.222.0.100 () {56 vars in 1381 bytes} [Mon Aug 1 15:00:25 2022] GET /identity/v3/role_assignments?group.id=67c25d20e3e34c0b9d74a852083437c0&scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80&role.id=d1f6973ce11a4644b649b38f8b2df173 => generated 261 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:25.218057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c189b3-1d53-4a8f-82f6-dc443d52a7ba None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:25.251453 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3ebbf-ab81-4444-ab6b-d8fc08475501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:25.252624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3ebbf-ab81-4444-ab6b-d8fc08475501 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:25.253171 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3ebbf-ab81-4444-ab6b-d8fc08475501 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:25.253608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3ebbf-ab81-4444-ab6b-d8fc08475501 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:25.254702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3ebbf-ab81-4444-ab6b-d8fc08475501 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:25.255170 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3ebbf-ab81-4444-ab6b-d8fc08475501 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:25.257876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3ebbf-ab81-4444-ab6b-d8fc08475501 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:25.270190 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 186/366] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:25 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:25.271582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3ebbf-ab81-4444-ab6b-d8fc08475501 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:25.374277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c189b3-1d53-4a8f-82f6-dc443d52a7ba None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:25.423969 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:25.424205 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99071587-bf3c-4ae6-b609-0b7e1d963efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:25.424976 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99071587-bf3c-4ae6-b609-0b7e1d963efd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:25.424976 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99071587-bf3c-4ae6-b609-0b7e1d963efd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:25.425218 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99071587-bf3c-4ae6-b609-0b7e1d963efd None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:25.426977 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99071587-bf3c-4ae6-b609-0b7e1d963efd None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:25.428691 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99071587-bf3c-4ae6-b609-0b7e1d963efd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:25.431245 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 181/367] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:25 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:25.432083 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99071587-bf3c-4ae6-b609-0b7e1d963efd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:25.447213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3ebbf-ab81-4444-ab6b-d8fc08475501 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:25.457623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:25.497951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5e38f-80f5-4fdc-bc97-38e5ff13e897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:25.499315 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5e38f-80f5-4fdc-bc97-38e5ff13e897 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:25.500120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5e38f-80f5-4fdc-bc97-38e5ff13e897 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:25.500663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5e38f-80f5-4fdc-bc97-38e5ff13e897 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:25.501790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5e38f-80f5-4fdc-bc97-38e5ff13e897 None admin] RBAC: Adding query filter params (role.id=471fd358281343cba97902b70b416f32, user.id=d0cfe09ac8dc4244b9f1117752c03e5a, scope.system=all) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:25.502376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5e38f-80f5-4fdc-bc97-38e5ff13e897 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:25.504751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc5e38f-80f5-4fdc-bc97-38e5ff13e897 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:25.530249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 187/368] 10.222.0.100 () {56 vars in 1313 bytes} [Mon Aug 1 15:00:25 2022] GET /identity/v3/role_assignments?user.id=d0cfe09ac8dc4244b9f1117752c03e5a&scope.system=all&role.id=471fd358281343cba97902b70b416f32 => generated 512 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:25.530249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5e38f-80f5-4fdc-bc97-38e5ff13e897 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:25.630631 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99071587-bf3c-4ae6-b609-0b7e1d963efd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:25.713556 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f78619-e191-492e-876f-0b9ddd74eb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:25.714578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78619-e191-492e-876f-0b9ddd74eb1c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:25.715130 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78619-e191-492e-876f-0b9ddd74eb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:25.715588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f78619-e191-492e-876f-0b9ddd74eb1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:25.747848 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f78619-e191-492e-876f-0b9ddd74eb1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-21f78619-e191-492e-876f-0b9ddd74eb1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:25.758869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:25.759980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:25.760840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:25.760840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:25.760840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:25.760840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:25.760840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:25.760840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:25.760840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:25.760840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:25.760840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:25.760840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:25.760840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:25.760840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:25.760840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 182/369] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:00:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:25.762012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f78619-e191-492e-876f-0b9ddd74eb1c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:25.792727 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5e38f-80f5-4fdc-bc97-38e5ff13e897 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:25.855948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf7d9a-b9b4-4fb8-8949-368dfe68c907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:25.856864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf7d9a-b9b4-4fb8-8949-368dfe68c907 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:25.857156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf7d9a-b9b4-4fb8-8949-368dfe68c907 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:25.857436 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf7d9a-b9b4-4fb8-8949-368dfe68c907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:25.879216 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:25.886244 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:25.918042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f78619-e191-492e-876f-0b9ddd74eb1c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:25.953245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf7d9a-b9b4-4fb8-8949-368dfe68c907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:25.954765 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf7d9a-b9b4-4fb8-8949-368dfe68c907 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:25.968983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39351ba2-c205-482a-a28e-3d8feb63250e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:25.969563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39351ba2-c205-482a-a28e-3d8feb63250e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:25.969844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39351ba2-c205-482a-a28e-3d8feb63250e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:25.970121 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39351ba2-c205-482a-a28e-3d8feb63250e None admin] PATH_INFO: `/v3/groups/67c25d20e3e34c0b9d74a852083437c0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:25.971341 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39351ba2-c205-482a-a28e-3d8feb63250e None admin] RBAC: Authorizing `identity:get_group(group_id=67c25d20e3e34c0b9d74a852083437c0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:25.972819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39351ba2-c205-482a-a28e-3d8feb63250e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:25.974533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 183/370] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:25 2022] GET /identity/v3/groups/67c25d20e3e34c0b9d74a852083437c0 => generated 227 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:25.975415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39351ba2-c205-482a-a28e-3d8feb63250e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:26.071013 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 188/371] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:25 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:26.071912 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf7d9a-b9b4-4fb8-8949-368dfe68c907 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:26.098358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39351ba2-c205-482a-a28e-3d8feb63250e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:26.124124 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:26.138024 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee38b2a-25fc-49e0-9ff8-8eaba35f1a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:26.138718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee38b2a-25fc-49e0-9ff8-8eaba35f1a9c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:26.139066 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee38b2a-25fc-49e0-9ff8-8eaba35f1a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:26.139355 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee38b2a-25fc-49e0-9ff8-8eaba35f1a9c None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:26.141107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee38b2a-25fc-49e0-9ff8-8eaba35f1a9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:26.141436 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee38b2a-25fc-49e0-9ff8-8eaba35f1a9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:26.143062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee38b2a-25fc-49e0-9ff8-8eaba35f1a9c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:26.153676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 184/372] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:25 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:26.154536 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee38b2a-25fc-49e0-9ff8-8eaba35f1a9c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:26.177069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:26.184699 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf7d9a-b9b4-4fb8-8949-368dfe68c907 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:26.224009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055d5b6-f8a4-49c0-b1e1-7367e5d8a6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:26.224610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055d5b6-f8a4-49c0-b1e1-7367e5d8a6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:26.224900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055d5b6-f8a4-49c0-b1e1-7367e5d8a6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:26.225180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055d5b6-f8a4-49c0-b1e1-7367e5d8a6e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:26.226016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055d5b6-f8a4-49c0-b1e1-7367e5d8a6e8 None admin] RBAC: Adding query filter params (group.id=67c25d20e3e34c0b9d74a852083437c0, role.id=d1f6973ce11a4644b649b38f8b2df173, scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:26.226336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055d5b6-f8a4-49c0-b1e1-7367e5d8a6e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:26.228854 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055d5b6-f8a4-49c0-b1e1-7367e5d8a6e8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:26.237147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 189/373] 10.222.0.100 () {56 vars in 1381 bytes} [Mon Aug 1 15:00:26 2022] GET /identity/v3/role_assignments?group.id=67c25d20e3e34c0b9d74a852083437c0&scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f&role.id=d1f6973ce11a4644b649b38f8b2df173 => generated 261 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:26.237908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055d5b6-f8a4-49c0-b1e1-7367e5d8a6e8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:26.312786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee38b2a-25fc-49e0-9ff8-8eaba35f1a9c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:26.355032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e5da2-6815-4875-b99f-3a67dcf1dde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:26.356116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e5da2-6815-4875-b99f-3a67dcf1dde9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:26.356645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e5da2-6815-4875-b99f-3a67dcf1dde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:26.357097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e5da2-6815-4875-b99f-3a67dcf1dde9 None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:26.358622 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e5da2-6815-4875-b99f-3a67dcf1dde9 None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:26.360425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e5da2-6815-4875-b99f-3a67dcf1dde9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:26.362902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 185/374] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:26 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:26.365837 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055d5b6-f8a4-49c0-b1e1-7367e5d8a6e8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:26.369235 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e5da2-6815-4875-b99f-3a67dcf1dde9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:26.375008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:26.405736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c431e6-182e-4b45-8dcc-ca062635f807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:26.406336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c431e6-182e-4b45-8dcc-ca062635f807 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:26.406626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c431e6-182e-4b45-8dcc-ca062635f807 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:26.406940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c431e6-182e-4b45-8dcc-ca062635f807 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:26.469403 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:26.488817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c431e6-182e-4b45-8dcc-ca062635f807 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:26.490334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c431e6-182e-4b45-8dcc-ca062635f807 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:26.512968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e5da2-6815-4875-b99f-3a67dcf1dde9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:26.576759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee089f7-db92-41e7-a292-d29a2b163242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:26.577700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee089f7-db92-41e7-a292-d29a2b163242 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:26.578127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee089f7-db92-41e7-a292-d29a2b163242 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:26.578552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee089f7-db92-41e7-a292-d29a2b163242 None admin] PATH_INFO: `/v3/groups/67c25d20e3e34c0b9d74a852083437c0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:26.580040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee089f7-db92-41e7-a292-d29a2b163242 None admin] RBAC: Authorizing `identity:get_group(group_id=67c25d20e3e34c0b9d74a852083437c0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:26.581751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee089f7-db92-41e7-a292-d29a2b163242 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:26.583997 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 186/375] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:26 2022] GET /identity/v3/groups/67c25d20e3e34c0b9d74a852083437c0 => generated 227 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:26.590760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee089f7-db92-41e7-a292-d29a2b163242 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:26.613749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 190/376] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:26 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:26.621098 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c431e6-182e-4b45-8dcc-ca062635f807 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:26.638834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:26.731510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee089f7-db92-41e7-a292-d29a2b163242 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:26.778010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c431e6-182e-4b45-8dcc-ca062635f807 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:26.784232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad182a4-2182-42b6-b9c4-17c42aa6b9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:26.785511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad182a4-2182-42b6-b9c4-17c42aa6b9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:26.785973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad182a4-2182-42b6-b9c4-17c42aa6b9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:26.786484 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad182a4-2182-42b6-b9c4-17c42aa6b9b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:26.787505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad182a4-2182-42b6-b9c4-17c42aa6b9b0 None admin] RBAC: Adding query filter params (group.id=67c25d20e3e34c0b9d74a852083437c0, role.id=57221d717a454c7fae4f2df1de9dbe5b, scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:26.787993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad182a4-2182-42b6-b9c4-17c42aa6b9b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:26.789706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad182a4-2182-42b6-b9c4-17c42aa6b9b0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:26.797825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 187/377] 10.222.0.100 () {56 vars in 1381 bytes} [Mon Aug 1 15:00:26 2022] GET /identity/v3/role_assignments?group.id=67c25d20e3e34c0b9d74a852083437c0&scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f&role.id=57221d717a454c7fae4f2df1de9dbe5b => generated 261 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:26.800367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed793612-dead-4338-98f3-2312d7e716d5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:26.800863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed793612-dead-4338-98f3-2312d7e716d5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:26.801342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed793612-dead-4338-98f3-2312d7e716d5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:26.808913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 188/378] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:26 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:26.812852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbd530-ddc2-4da3-ae1e-0e150152d27a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:26.813212 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbd530-ddc2-4da3-ae1e-0e150152d27a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:26.813520 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbd530-ddc2-4da3-ae1e-0e150152d27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:26.829796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:26.836362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-68cbd530-ddc2-4da3-ae1e-0e150152d27a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:26.837428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-68cbd530-ddc2-4da3-ae1e-0e150152d27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:26.843395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fd411-9080-486e-9ad5-b597221084ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:26.844430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fd411-9080-486e-9ad5-b597221084ac None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:26.844910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fd411-9080-486e-9ad5-b597221084ac None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:26.845375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fd411-9080-486e-9ad5-b597221084ac None admin] PATH_INFO: `/v3/projects/bcc1ff25500841a3a7180e5fff03ff9f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:26.847340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fd411-9080-486e-9ad5-b597221084ac None admin] RBAC: Authorizing `identity:get_project(project_id=bcc1ff25500841a3a7180e5fff03ff9f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:26.849553 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fd411-9080-486e-9ad5-b597221084ac None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:26.853178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 191/379] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:26 2022] GET /identity/v3/projects/bcc1ff25500841a3a7180e5fff03ff9f => generated 345 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:26.859240 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fd411-9080-486e-9ad5-b597221084ac None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:26.980087 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 189/380] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:27.010372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:27.027814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbd530-ddc2-4da3-ae1e-0e150152d27a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:27.063469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fd411-9080-486e-9ad5-b597221084ac None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:27.125940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87457859-acde-4388-9199-b7ec2afa2f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:27.127279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87457859-acde-4388-9199-b7ec2afa2f88 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:27.127761 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87457859-acde-4388-9199-b7ec2afa2f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:27.128254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87457859-acde-4388-9199-b7ec2afa2f88 None admin] PATH_INFO: `/v3/groups/17547fbfbfc2476e9eda70a0eedcd2a0` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:27.140377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87457859-acde-4388-9199-b7ec2afa2f88 None admin] RBAC: Authorizing `identity:get_group(group_id=17547fbfbfc2476e9eda70a0eedcd2a0)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:27.141603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87457859-acde-4388-9199-b7ec2afa2f88 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:27.143708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 192/381] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:26 2022] GET /identity/v3/groups/17547fbfbfc2476e9eda70a0eedcd2a0 => generated 230 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:27.161597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87457859-acde-4388-9199-b7ec2afa2f88 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:27.182371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbd530-ddc2-4da3-ae1e-0e150152d27a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:27.228944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457168b-ed41-4af7-ba8e-75d407657be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:27.229965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457168b-ed41-4af7-ba8e-75d407657be6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:27.230438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457168b-ed41-4af7-ba8e-75d407657be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:27.230968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9457168b-ed41-4af7-ba8e-75d407657be6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:27.234598 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:27.247521 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9457168b-ed41-4af7-ba8e-75d407657be6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9457168b-ed41-4af7-ba8e-75d407657be6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:27.254297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:27.255265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:27.256078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:27.256078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:27.256078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:27.256078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:27.256078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:27.256078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:27.256078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:27.256078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:27.256078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:27.256078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:27.256078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:27.256078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:27.259105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 190/382] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:00:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:27.266062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457168b-ed41-4af7-ba8e-75d407657be6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:27.338372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87457859-acde-4388-9199-b7ec2afa2f88 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:27.384820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4148fd0-b577-4466-8213-e01a69643e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:27.385464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4148fd0-b577-4466-8213-e01a69643e02 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:27.385754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4148fd0-b577-4466-8213-e01a69643e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:27.389574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4148fd0-b577-4466-8213-e01a69643e02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:27.390542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4148fd0-b577-4466-8213-e01a69643e02 None admin] RBAC: Adding query filter params (group.id=17547fbfbfc2476e9eda70a0eedcd2a0, role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=bcc1ff25500841a3a7180e5fff03ff9f) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:27.391078 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4148fd0-b577-4466-8213-e01a69643e02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:27.393488 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4148fd0-b577-4466-8213-e01a69643e02 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:27.407120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 193/383] 10.222.0.100 () {56 vars in 1381 bytes} [Mon Aug 1 15:00:27 2022] GET /identity/v3/role_assignments?group.id=17547fbfbfc2476e9eda70a0eedcd2a0&scope.project.id=bcc1ff25500841a3a7180e5fff03ff9f&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 261 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:27.411268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9457168b-ed41-4af7-ba8e-75d407657be6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:27.465097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d03e46-5faa-4d9e-af96-0e6531cda2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:27.466087 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d03e46-5faa-4d9e-af96-0e6531cda2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:27.466583 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d03e46-5faa-4d9e-af96-0e6531cda2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:27.467087 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d03e46-5faa-4d9e-af96-0e6531cda2b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:27.468026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d03e46-5faa-4d9e-af96-0e6531cda2b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:27.468742 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d03e46-5faa-4d9e-af96-0e6531cda2b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:27.470425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d03e46-5faa-4d9e-af96-0e6531cda2b7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:27.480329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 191/384] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:27 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:27.485969 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4148fd0-b577-4466-8213-e01a69643e02 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:27.490919 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:27.674690 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4148fd0-b577-4466-8213-e01a69643e02 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:27.718352 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b2804-c44a-4c7a-8dda-10efd7f42417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:27.718900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b2804-c44a-4c7a-8dda-10efd7f42417 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:27.719187 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b2804-c44a-4c7a-8dda-10efd7f42417 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:27.719470 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b2804-c44a-4c7a-8dda-10efd7f42417 None admin] PATH_INFO: `/v3/groups/67c25d20e3e34c0b9d74a852083437c0` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:27.720664 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b2804-c44a-4c7a-8dda-10efd7f42417 None admin] RBAC: Authorizing `identity:get_group(group_id=67c25d20e3e34c0b9d74a852083437c0)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:27.722109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b2804-c44a-4c7a-8dda-10efd7f42417 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:27.723963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 194/385] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:27 2022] GET /identity/v3/groups/67c25d20e3e34c0b9d74a852083437c0 => generated 227 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:27.729893 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d03e46-5faa-4d9e-af96-0e6531cda2b7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:27.820314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:27.868120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d03e46-5faa-4d9e-af96-0e6531cda2b7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:27.915429 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21167b6e-de92-4863-980d-0f9fadf55f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:27.916485 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21167b6e-de92-4863-980d-0f9fadf55f77 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:27.917108 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21167b6e-de92-4863-980d-0f9fadf55f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:27.917600 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21167b6e-de92-4863-980d-0f9fadf55f77 None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:27.919103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21167b6e-de92-4863-980d-0f9fadf55f77 None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:27.920798 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21167b6e-de92-4863-980d-0f9fadf55f77 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:27.923401 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 192/386] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:27 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:27.928983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b2804-c44a-4c7a-8dda-10efd7f42417 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:28.043144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:28.083143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b2804-c44a-4c7a-8dda-10efd7f42417 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:28.132130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133adac-d52b-44c4-8969-41958ae4add6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:28.132836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133adac-d52b-44c4-8969-41958ae4add6 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:28.133123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133adac-d52b-44c4-8969-41958ae4add6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:28.133430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133adac-d52b-44c4-8969-41958ae4add6 None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80/groups/67c25d20e3e34c0b9d74a852083437c0/roles/57221d717a454c7fae4f2df1de9dbe5b` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:28.135764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133adac-d52b-44c4-8969-41958ae4add6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80, group_id=67c25d20e3e34c0b9d74a852083437c0, role_id=57221d717a454c7fae4f2df1de9dbe5b)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:28.137251 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133adac-d52b-44c4-8969-41958ae4add6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:28.155024 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 195/387] 10.222.0.100 () {58 vars in 1447 bytes} [Mon Aug 1 15:00:27 2022] PUT /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80/groups/67c25d20e3e34c0b9d74a852083437c0/roles/57221d717a454c7fae4f2df1de9dbe5b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:28.392959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5199e-8047-4e2e-82eb-c57d77cdbe3a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:28.393370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5199e-8047-4e2e-82eb-c57d77cdbe3a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:28.393553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5199e-8047-4e2e-82eb-c57d77cdbe3a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:28.395381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 193/388] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:28.401140 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd825b4d-860d-4322-8f6a-319dbef7fd4d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:28.401434 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd825b4d-860d-4322-8f6a-319dbef7fd4d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:28.401718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd825b4d-860d-4322-8f6a-319dbef7fd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:28.431213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-cd825b4d-860d-4322-8f6a-319dbef7fd4d None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:28.431213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-cd825b4d-860d-4322-8f6a-319dbef7fd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:28.513116 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 196/389] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:28.536141 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5199e-8047-4e2e-82eb-c57d77cdbe3a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:28.675463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff5199e-8047-4e2e-82eb-c57d77cdbe3a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:28.712382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:28.727055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4257991-f80a-4ec4-8a18-af8646ccef86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:28.727711 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4257991-f80a-4ec4-8a18-af8646ccef86 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:28.727994 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4257991-f80a-4ec4-8a18-af8646ccef86 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:28.728273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4257991-f80a-4ec4-8a18-af8646ccef86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:28.744609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4257991-f80a-4ec4-8a18-af8646ccef86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-d4257991-f80a-4ec4-8a18-af8646ccef86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:28.750151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:28.751102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:28.751899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:28.751899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:28.751899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:28.751899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:28.751899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:28.751899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:28.751899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:28.751899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:28.751899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:28.751899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:28.751899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:28.751899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:28.754025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 194/390] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:28.759798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd825b4d-860d-4322-8f6a-319dbef7fd4d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:28.844090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:28.894898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd825b4d-860d-4322-8f6a-319dbef7fd4d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:28.942412 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e60e1-7dc4-4464-8ee6-74330aab5dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:28.943407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e60e1-7dc4-4464-8ee6-74330aab5dfa None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:28.943911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e60e1-7dc4-4464-8ee6-74330aab5dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:28.944489 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e60e1-7dc4-4464-8ee6-74330aab5dfa None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:28.945419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e60e1-7dc4-4464-8ee6-74330aab5dfa None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:28.945938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e60e1-7dc4-4464-8ee6-74330aab5dfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:28.947613 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22e60e1-7dc4-4464-8ee6-74330aab5dfa None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:28.957725 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 197/391] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:28 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:28.964925 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4257991-f80a-4ec4-8a18-af8646ccef86 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:29.098896 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce788f9d-8c37-48ab-a836-eec2e014b4cf None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:29.099408 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce788f9d-8c37-48ab-a836-eec2e014b4cf None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:29.099485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce788f9d-8c37-48ab-a836-eec2e014b4cf None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:29.101350 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 198/392] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:29.108197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bc388-63e9-4713-9b56-fd0222b8bd02 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:29.108782 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bc388-63e9-4713-9b56-fd0222b8bd02 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:29.109262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bc388-63e9-4713-9b56-fd0222b8bd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:29.128281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-245bc388-63e9-4713-9b56-fd0222b8bd02 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:29.129844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-245bc388-63e9-4713-9b56-fd0222b8bd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:29.151326 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4257991-f80a-4ec4-8a18-af8646ccef86 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:29.215022 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8776be-cfdf-4e39-b6b0-29195f4a37bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:29.215590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8776be-cfdf-4e39-b6b0-29195f4a37bf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:29.215876 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8776be-cfdf-4e39-b6b0-29195f4a37bf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:29.216153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8776be-cfdf-4e39-b6b0-29195f4a37bf None admin] PATH_INFO: `/v3/groups/67c25d20e3e34c0b9d74a852083437c0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:29.217401 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8776be-cfdf-4e39-b6b0-29195f4a37bf None admin] RBAC: Authorizing `identity:get_group(group_id=67c25d20e3e34c0b9d74a852083437c0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:29.220943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8776be-cfdf-4e39-b6b0-29195f4a37bf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:29.224406 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 195/393] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:28 2022] GET /identity/v3/groups/67c25d20e3e34c0b9d74a852083437c0 => generated 227 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:29.230596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8776be-cfdf-4e39-b6b0-29195f4a37bf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:29.244733 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 199/394] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:29.300454 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bc388-63e9-4713-9b56-fd0222b8bd02 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:29.301318 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:29.402506 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8776be-cfdf-4e39-b6b0-29195f4a37bf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:29.419035 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:29.462037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef03df5-91ec-4b45-b96f-a6f71d185139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:29.463377 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef03df5-91ec-4b45-b96f-a6f71d185139 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:29.463729 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef03df5-91ec-4b45-b96f-a6f71d185139 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:29.464007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef03df5-91ec-4b45-b96f-a6f71d185139 None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:29.467105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef03df5-91ec-4b45-b96f-a6f71d185139 None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:29.468527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef03df5-91ec-4b45-b96f-a6f71d185139 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:29.472027 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 196/395] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:29 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:29.477011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef03df5-91ec-4b45-b96f-a6f71d185139 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:29.509749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bc388-63e9-4713-9b56-fd0222b8bd02 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:29.572393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f804db-d4ab-4620-a111-022373464ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:29.573361 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f804db-d4ab-4620-a111-022373464ceb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:29.573764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f804db-d4ab-4620-a111-022373464ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:29.574252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f804db-d4ab-4620-a111-022373464ceb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:29.590132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f804db-d4ab-4620-a111-022373464ceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f8f804db-d4ab-4620-a111-022373464ceb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:29.595616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:29.596530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:29.598008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:29.598008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:29.598008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:29.598008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:29.598008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:29.598008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:29.598008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:29.598008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:29.598008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:29.598008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:29.598008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:29.598008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:29.598008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 200/396] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:29.598916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c267d37-9266-476e-8296-06c0b6c38604 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:29.599260 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c267d37-9266-476e-8296-06c0b6c38604 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:29.601200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c267d37-9266-476e-8296-06c0b6c38604 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:29.605175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 201/397] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:29.606063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c267d37-9266-476e-8296-06c0b6c38604 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:29.637244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef03df5-91ec-4b45-b96f-a6f71d185139 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:29.663176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:29.676975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e26653-e7dd-4ca8-8bc0-9446e892e5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:29.678062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e26653-e7dd-4ca8-8bc0-9446e892e5ba None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:29.678557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e26653-e7dd-4ca8-8bc0-9446e892e5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:29.679623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e26653-e7dd-4ca8-8bc0-9446e892e5ba None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80/groups/67c25d20e3e34c0b9d74a852083437c0/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:29.683077 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e26653-e7dd-4ca8-8bc0-9446e892e5ba None admin] RBAC: Authorizing `identity:create_grant(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80, group_id=67c25d20e3e34c0b9d74a852083437c0, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:29.684883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e26653-e7dd-4ca8-8bc0-9446e892e5ba None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:29.702309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 197/398] 10.222.0.100 () {58 vars in 1447 bytes} [Mon Aug 1 15:00:29 2022] PUT /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80/groups/67c25d20e3e34c0b9d74a852083437c0/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:29.704639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e496a1-1863-4f02-80a5-6e23af52e4b2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:29.705402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e496a1-1863-4f02-80a5-6e23af52e4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:29.706079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e496a1-1863-4f02-80a5-6e23af52e4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:29.729759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e8e496a1-1863-4f02-80a5-6e23af52e4b2 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:29.731588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e8e496a1-1863-4f02-80a5-6e23af52e4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:29.759499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c267d37-9266-476e-8296-06c0b6c38604 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:29.786105 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:29.827276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 198/399] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:29.865530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019926a-3113-453b-91cc-a7030ae774aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:29.866228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019926a-3113-453b-91cc-a7030ae774aa None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:29.866521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019926a-3113-453b-91cc-a7030ae774aa None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:29.866875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019926a-3113-453b-91cc-a7030ae774aa None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:29.867223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e496a1-1863-4f02-80a5-6e23af52e4b2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:29.867623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019926a-3113-453b-91cc-a7030ae774aa None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:29.867932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019926a-3113-453b-91cc-a7030ae774aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:29.869362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019926a-3113-453b-91cc-a7030ae774aa None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:29.891101 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 202/400] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:29 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:29.896945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019926a-3113-453b-91cc-a7030ae774aa None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:30.044114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e496a1-1863-4f02-80a5-6e23af52e4b2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:30.056045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019926a-3113-453b-91cc-a7030ae774aa None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:30.061478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:30.089509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a0422-6c6b-4ef7-af59-69dfe323224f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:30.091205 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0422-6c6b-4ef7-af59-69dfe323224f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:30.091696 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0422-6c6b-4ef7-af59-69dfe323224f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:30.092115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0422-6c6b-4ef7-af59-69dfe323224f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:30.103246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7accf4-4235-49ba-92b4-b4932bece4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:30.103892 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7accf4-4235-49ba-92b4-b4932bece4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:30.104181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7accf4-4235-49ba-92b4-b4932bece4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:30.104461 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7accf4-4235-49ba-92b4-b4932bece4f7 None admin] PATH_INFO: `/v3/groups/67c25d20e3e34c0b9d74a852083437c0` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:30.105682 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7accf4-4235-49ba-92b4-b4932bece4f7 None admin] RBAC: Authorizing `identity:get_group(group_id=67c25d20e3e34c0b9d74a852083437c0)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:30.107074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a0422-6c6b-4ef7-af59-69dfe323224f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:30.107515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7accf4-4235-49ba-92b4-b4932bece4f7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:30.109455 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 203/401] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:29 2022] GET /identity/v3/groups/67c25d20e3e34c0b9d74a852083437c0 => generated 227 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:30.114694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49dd067-d9f4-4212-9426-b84f54dc8ac0 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:30.114694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49dd067-d9f4-4212-9426-b84f54dc8ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:30.114694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49dd067-d9f4-4212-9426-b84f54dc8ac0 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:30.114694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 204/402] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-eb1a0422-6c6b-4ef7-af59-69dfe323224f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:30.118928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:30.119799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:30.120595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:30.120595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:30.120595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:30.120595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:30.120595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:30.120595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:30.120595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:30.120595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:30.120595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:30.120595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:30.120595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:30.120595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:30.120595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 199/403] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:00:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:30.123681 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a0422-6c6b-4ef7-af59-69dfe323224f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:30.124163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d8806-7c37-4e8c-b428-a48e58b1740a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:30.124474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d8806-7c37-4e8c-b428-a48e58b1740a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:30.124887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d8806-7c37-4e8c-b428-a48e58b1740a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:30.150322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1d8d8806-7c37-4e8c-b428-a48e58b1740a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:30.152225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1d8d8806-7c37-4e8c-b428-a48e58b1740a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:30.216749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:30.246935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a0422-6c6b-4ef7-af59-69dfe323224f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:30.260912 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 205/404] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:30.260912 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d8806-7c37-4e8c-b428-a48e58b1740a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:30.312183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa4c8b-9142-4b76-be1e-413ddad63ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:30.313195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4c8b-9142-4b76-be1e-413ddad63ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:30.313699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4c8b-9142-4b76-be1e-413ddad63ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:30.314169 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa4c8b-9142-4b76-be1e-413ddad63ae1 None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:30.315657 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa4c8b-9142-4b76-be1e-413ddad63ae1 None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:30.317426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa4c8b-9142-4b76-be1e-413ddad63ae1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:30.319487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 200/405] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:30 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:30.322083 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa4c8b-9142-4b76-be1e-413ddad63ae1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:30.368612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:30.390005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d8806-7c37-4e8c-b428-a48e58b1740a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:30.430384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef202522-e994-4113-9c89-0515907666aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:30.431062 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef202522-e994-4113-9c89-0515907666aa None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:30.431348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef202522-e994-4113-9c89-0515907666aa None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:30.431624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef202522-e994-4113-9c89-0515907666aa None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:30.432347 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef202522-e994-4113-9c89-0515907666aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:30.432655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef202522-e994-4113-9c89-0515907666aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:30.434047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef202522-e994-4113-9c89-0515907666aa None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:30.440842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa4c8b-9142-4b76-be1e-413ddad63ae1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:30.448517 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 206/406] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:30 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:30.448517 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef202522-e994-4113-9c89-0515907666aa None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:30.489662 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48aec1f-2bdb-4036-bca1-363a3f0badab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:30.490335 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48aec1f-2bdb-4036-bca1-363a3f0badab None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:30.490650 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48aec1f-2bdb-4036-bca1-363a3f0badab None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:30.490967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48aec1f-2bdb-4036-bca1-363a3f0badab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:30.496631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:30.509487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48aec1f-2bdb-4036-bca1-363a3f0badab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a48aec1f-2bdb-4036-bca1-363a3f0badab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:30.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:30.516975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:30.517884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:30.517884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:30.517884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:30.517884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:30.517884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:30.517884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:30.517884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:30.517884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:30.517884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:30.517884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:30.517884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:30.517884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:30.526404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 201/407] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:00:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:30.526404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48aec1f-2bdb-4036-bca1-363a3f0badab None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:30.589334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef202522-e994-4113-9c89-0515907666aa None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:30.636318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912f401-a4ed-4fd0-803f-3bf87ea25773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:30.637412 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912f401-a4ed-4fd0-803f-3bf87ea25773 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:30.637843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912f401-a4ed-4fd0-803f-3bf87ea25773 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:30.638266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912f401-a4ed-4fd0-803f-3bf87ea25773 None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f/groups/67c25d20e3e34c0b9d74a852083437c0/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:30.641030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912f401-a4ed-4fd0-803f-3bf87ea25773 None admin] RBAC: Authorizing `identity:create_grant(project_id=ebb7ad89c9734ecbbd41c0c2de27069f, group_id=67c25d20e3e34c0b9d74a852083437c0, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:30.642498 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:30.642634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912f401-a4ed-4fd0-803f-3bf87ea25773 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:30.647288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48aec1f-2bdb-4036-bca1-363a3f0badab None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:30.659482 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 207/408] 10.222.0.100 () {58 vars in 1447 bytes} [Mon Aug 1 15:00:30 2022] PUT /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f/groups/67c25d20e3e34c0b9d74a852083437c0/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:30.661169 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912f401-a4ed-4fd0-803f-3bf87ea25773 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:30.689652 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cafff9-34c0-43b5-a1a2-0399eabbd0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:30.690768 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cafff9-34c0-43b5-a1a2-0399eabbd0ba None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:30.691182 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cafff9-34c0-43b5-a1a2-0399eabbd0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:30.691524 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cafff9-34c0-43b5-a1a2-0399eabbd0ba None admin] PATH_INFO: `/v3/groups/67c25d20e3e34c0b9d74a852083437c0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:30.693012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cafff9-34c0-43b5-a1a2-0399eabbd0ba None admin] RBAC: Authorizing `identity:get_group(group_id=67c25d20e3e34c0b9d74a852083437c0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:30.695128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cafff9-34c0-43b5-a1a2-0399eabbd0ba None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:30.697207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 202/409] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:30 2022] GET /identity/v3/groups/67c25d20e3e34c0b9d74a852083437c0 => generated 227 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:30.702872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cafff9-34c0-43b5-a1a2-0399eabbd0ba None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:30.782925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:30.793691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912f401-a4ed-4fd0-803f-3bf87ea25773 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:30.837543 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92749c42-1358-4d2f-9651-f85f938e1b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:30.838874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92749c42-1358-4d2f-9651-f85f938e1b00 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:30.839397 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92749c42-1358-4d2f-9651-f85f938e1b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:30.839856 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92749c42-1358-4d2f-9651-f85f938e1b00 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:30.840768 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92749c42-1358-4d2f-9651-f85f938e1b00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:30.841258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92749c42-1358-4d2f-9651-f85f938e1b00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:30.842945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92749c42-1358-4d2f-9651-f85f938e1b00 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:30.853795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 208/410] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:30 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:30.863894 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92749c42-1358-4d2f-9651-f85f938e1b00 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:30.878301 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cafff9-34c0-43b5-a1a2-0399eabbd0ba None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:30.916411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa5f2a-ecf3-477d-afc1-4098328eba0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:30.917133 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa5f2a-ecf3-477d-afc1-4098328eba0f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:30.917414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa5f2a-ecf3-477d-afc1-4098328eba0f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:30.917690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa5f2a-ecf3-477d-afc1-4098328eba0f None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:30.918921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa5f2a-ecf3-477d-afc1-4098328eba0f None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:30.920460 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa5f2a-ecf3-477d-afc1-4098328eba0f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:30.922756 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 203/411] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:30 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:30.931840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa5f2a-ecf3-477d-afc1-4098328eba0f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:30.989892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:31.031635 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92749c42-1358-4d2f-9651-f85f938e1b00 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:31.073925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a49c3-f851-47a9-b6e3-9226757a1120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:31.074575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a49c3-f851-47a9-b6e3-9226757a1120 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:31.074942 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a49c3-f851-47a9-b6e3-9226757a1120 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:31.075222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a49c3-f851-47a9-b6e3-9226757a1120 None admin] PATH_INFO: `/v3/groups/17547fbfbfc2476e9eda70a0eedcd2a0` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:31.076388 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a49c3-f851-47a9-b6e3-9226757a1120 None admin] RBAC: Authorizing `identity:get_group(group_id=17547fbfbfc2476e9eda70a0eedcd2a0)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:31.078237 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a49c3-f851-47a9-b6e3-9226757a1120 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:31.085155 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 209/412] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:30 2022] GET /identity/v3/groups/17547fbfbfc2476e9eda70a0eedcd2a0 => generated 230 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:31.086891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9aa72c-a36d-474a-ad21-07673bc585ed None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:31.087182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9aa72c-a36d-474a-ad21-07673bc585ed None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:31.087586 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9aa72c-a36d-474a-ad21-07673bc585ed None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:31.089117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa5f2a-ecf3-477d-afc1-4098328eba0f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:31.089817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 210/413] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:31 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:31.095405 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9aa72c-a36d-474a-ad21-07673bc585ed None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:31.128686 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:31.137431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f4543-dd47-47f3-aee7-621cf051ed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:31.138367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4543-dd47-47f3-aee7-621cf051ed3c None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:31.138903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4543-dd47-47f3-aee7-621cf051ed3c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:31.139342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4543-dd47-47f3-aee7-621cf051ed3c None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f/groups/67c25d20e3e34c0b9d74a852083437c0/roles/57221d717a454c7fae4f2df1de9dbe5b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:31.141903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f4543-dd47-47f3-aee7-621cf051ed3c None admin] RBAC: Authorizing `identity:create_grant(project_id=ebb7ad89c9734ecbbd41c0c2de27069f, group_id=67c25d20e3e34c0b9d74a852083437c0, role_id=57221d717a454c7fae4f2df1de9dbe5b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:31.143564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f4543-dd47-47f3-aee7-621cf051ed3c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:31.160268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 204/414] 10.222.0.100 () {58 vars in 1447 bytes} [Mon Aug 1 15:00:30 2022] PUT /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f/groups/67c25d20e3e34c0b9d74a852083437c0/roles/57221d717a454c7fae4f2df1de9dbe5b => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:31.168322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977ed18-001f-4c51-b901-269283b00f82 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:31.168938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977ed18-001f-4c51-b901-269283b00f82 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:31.169362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977ed18-001f-4c51-b901-269283b00f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:31.208064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9977ed18-001f-4c51-b901-269283b00f82 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:31.209526 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9977ed18-001f-4c51-b901-269283b00f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:31.246732 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9aa72c-a36d-474a-ad21-07673bc585ed None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:31.301262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89802417-6f00-436d-ad32-afec00e3e6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:31.301934 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89802417-6f00-436d-ad32-afec00e3e6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:31.302214 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89802417-6f00-436d-ad32-afec00e3e6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:31.302488 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89802417-6f00-436d-ad32-afec00e3e6d3 None admin] PATH_INFO: `/v3/projects/bcc1ff25500841a3a7180e5fff03ff9f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:31.305945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89802417-6f00-436d-ad32-afec00e3e6d3 None admin] RBAC: Authorizing `identity:get_project(project_id=bcc1ff25500841a3a7180e5fff03ff9f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:31.307496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89802417-6f00-436d-ad32-afec00e3e6d3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:31.309461 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 211/415] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:31 2022] GET /identity/v3/projects/bcc1ff25500841a3a7180e5fff03ff9f => generated 345 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:31.320707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89802417-6f00-436d-ad32-afec00e3e6d3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:31.378601 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 205/416] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:31.414378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977ed18-001f-4c51-b901-269283b00f82 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:31.488078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:31.525185 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:31.539424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89802417-6f00-436d-ad32-afec00e3e6d3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:31.610701 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b111af-ffca-4f96-b479-50ef411e648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:31.613077 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b111af-ffca-4f96-b479-50ef411e648c None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:31.613588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b111af-ffca-4f96-b479-50ef411e648c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:31.615060 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b111af-ffca-4f96-b479-50ef411e648c None admin] PATH_INFO: `/v3/projects/bcc1ff25500841a3a7180e5fff03ff9f/groups/17547fbfbfc2476e9eda70a0eedcd2a0/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:31.621158 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b111af-ffca-4f96-b479-50ef411e648c None admin] RBAC: Authorizing `identity:create_grant(project_id=bcc1ff25500841a3a7180e5fff03ff9f, group_id=17547fbfbfc2476e9eda70a0eedcd2a0, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:31.623498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b111af-ffca-4f96-b479-50ef411e648c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:31.646461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977ed18-001f-4c51-b901-269283b00f82 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:31.649658 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 212/417] 10.222.0.100 () {58 vars in 1447 bytes} [Mon Aug 1 15:00:31 2022] PUT /identity/v3/projects/bcc1ff25500841a3a7180e5fff03ff9f/groups/17547fbfbfc2476e9eda70a0eedcd2a0/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:31.696488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670413b4-ec6f-47a5-a20f-822cfd7b801c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:31.697612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670413b4-ec6f-47a5-a20f-822cfd7b801c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:31.698151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670413b4-ec6f-47a5-a20f-822cfd7b801c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:31.698681 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670413b4-ec6f-47a5-a20f-822cfd7b801c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:31.712590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670413b4-ec6f-47a5-a20f-822cfd7b801c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-670413b4-ec6f-47a5-a20f-822cfd7b801c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:31.718473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:31.719476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:31.720275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:31.720275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:31.720275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:31.720275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:31.720275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:31.720275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:31.720275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:31.720275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:31.720275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:31.720275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:31.720275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:31.720275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:31.723344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 206/418] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:00:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:31.729964 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b111af-ffca-4f96-b479-50ef411e648c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:31.874239 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b111af-ffca-4f96-b479-50ef411e648c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:31.931146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fadf7b-a7e8-45ea-a5cf-00ff4ed0a974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:31.932107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fadf7b-a7e8-45ea-a5cf-00ff4ed0a974 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:31.932585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fadf7b-a7e8-45ea-a5cf-00ff4ed0a974 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:31.933095 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fadf7b-a7e8-45ea-a5cf-00ff4ed0a974 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:31.934007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fadf7b-a7e8-45ea-a5cf-00ff4ed0a974 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:31.934510 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fadf7b-a7e8-45ea-a5cf-00ff4ed0a974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:31.936846 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fadf7b-a7e8-45ea-a5cf-00ff4ed0a974 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:31.953721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 213/419] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:31 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:31.959449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670413b4-ec6f-47a5-a20f-822cfd7b801c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:32.071061 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:32.085949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670413b4-ec6f-47a5-a20f-822cfd7b801c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:32.132617 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f7a62-7320-4a9e-8a67-12c444b1fe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:32.133260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f7a62-7320-4a9e-8a67-12c444b1fe94 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:32.133522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f7a62-7320-4a9e-8a67-12c444b1fe94 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:32.133802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f7a62-7320-4a9e-8a67-12c444b1fe94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:32.228563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f7a62-7320-4a9e-8a67-12c444b1fe94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:32.231049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bf2e0-d7d0-44a3-a620-a2715af162ff None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:32.231506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bf2e0-d7d0-44a3-a620-a2715af162ff None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:32.231579 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8bf2e0-d7d0-44a3-a620-a2715af162ff None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:32.232936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 214/420] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:32.233456 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f7a62-7320-4a9e-8a67-12c444b1fe94 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:32.238446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c46948-9bb0-47f9-92a7-9da736d2dffc None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:32.238863 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c46948-9bb0-47f9-92a7-9da736d2dffc None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:32.239015 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c46948-9bb0-47f9-92a7-9da736d2dffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:32.277068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e5c46948-9bb0-47f9-92a7-9da736d2dffc None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:32.287033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e5c46948-9bb0-47f9-92a7-9da736d2dffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:32.314962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:32.361663 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:32.396544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 207/421] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:31 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:32.402933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f7a62-7320-4a9e-8a67-12c444b1fe94 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:32.407404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 215/422] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:00:32.479083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c46948-9bb0-47f9-92a7-9da736d2dffc None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:32.545075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f7a62-7320-4a9e-8a67-12c444b1fe94 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:32.590532 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1d683-0348-4165-9a2a-9ff29cd613a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:32.591227 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1d683-0348-4165-9a2a-9ff29cd613a7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:32.591513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1d683-0348-4165-9a2a-9ff29cd613a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:32.591835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1d683-0348-4165-9a2a-9ff29cd613a7 None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:32.595474 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1d683-0348-4165-9a2a-9ff29cd613a7 None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:32.596955 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1d683-0348-4165-9a2a-9ff29cd613a7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:32.599010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 208/423] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:32 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:32.605058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1d683-0348-4165-9a2a-9ff29cd613a7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:32.666092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:32.716416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1d683-0348-4165-9a2a-9ff29cd613a7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:32.745608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c46948-9bb0-47f9-92a7-9da736d2dffc None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:32.755175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834167f4-ae1c-430a-a2ca-54f1f98efbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:32.756134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834167f4-ae1c-430a-a2ca-54f1f98efbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:32.756558 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834167f4-ae1c-430a-a2ca-54f1f98efbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:32.757635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834167f4-ae1c-430a-a2ca-54f1f98efbf8 None admin] PATH_INFO: `/v3/groups/67c25d20e3e34c0b9d74a852083437c0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:32.759027 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834167f4-ae1c-430a-a2ca-54f1f98efbf8 None admin] RBAC: Authorizing `identity:get_group(group_id=67c25d20e3e34c0b9d74a852083437c0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:32.760671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834167f4-ae1c-430a-a2ca-54f1f98efbf8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:32.762799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 209/424] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:32 2022] GET /identity/v3/groups/67c25d20e3e34c0b9d74a852083437c0 => generated 227 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:32.769179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834167f4-ae1c-430a-a2ca-54f1f98efbf8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:32.837046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecd269-4629-4b7a-b673-bdb032420dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:32.837824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecd269-4629-4b7a-b673-bdb032420dbb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:32.838610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecd269-4629-4b7a-b673-bdb032420dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:32.838991 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecd269-4629-4b7a-b673-bdb032420dbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:32.863990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adecd269-4629-4b7a-b673-bdb032420dbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-adecd269-4629-4b7a-b673-bdb032420dbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:32.873823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:32.875034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:32.875834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:32.875834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:32.875834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:32.875834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:32.875834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:32.875834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:32.875834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:32.875834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:32.875834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:32.875834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:32.875834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:32.875834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:32.883428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 216/425] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:32.883950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834167f4-ae1c-430a-a2ca-54f1f98efbf8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:32.889383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecd269-4629-4b7a-b673-bdb032420dbb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:32.908087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:32.917496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:32.925092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7293114-ca50-4c80-8809-126dc9e106de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:32.926008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7293114-ca50-4c80-8809-126dc9e106de None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:32.926464 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7293114-ca50-4c80-8809-126dc9e106de None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:32.927298 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7293114-ca50-4c80-8809-126dc9e106de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:32.929615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7293114-ca50-4c80-8809-126dc9e106de None admin] RBAC: Adding query filter params (group.id=67c25d20e3e34c0b9d74a852083437c0, role.id=57221d717a454c7fae4f2df1de9dbe5b, scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:32.930087 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7293114-ca50-4c80-8809-126dc9e106de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:32.931770 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7293114-ca50-4c80-8809-126dc9e106de None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:32.940175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 210/426] 10.222.0.100 () {56 vars in 1381 bytes} [Mon Aug 1 15:00:32 2022] GET /identity/v3/role_assignments?group.id=67c25d20e3e34c0b9d74a852083437c0&scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80&role.id=57221d717a454c7fae4f2df1de9dbe5b => generated 613 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:33.069513 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecd269-4629-4b7a-b673-bdb032420dbb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:33.125856 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b4be21-b97d-466f-aa83-7e076f2d923a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:33.126863 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4be21-b97d-466f-aa83-7e076f2d923a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:33.127317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4be21-b97d-466f-aa83-7e076f2d923a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:33.127745 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b4be21-b97d-466f-aa83-7e076f2d923a None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:33.128618 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b4be21-b97d-466f-aa83-7e076f2d923a None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:33.129068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b4be21-b97d-466f-aa83-7e076f2d923a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:33.130615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b4be21-b97d-466f-aa83-7e076f2d923a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:33.140775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 217/427] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:32 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:33.147123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7293114-ca50-4c80-8809-126dc9e106de None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:33.278389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7293114-ca50-4c80-8809-126dc9e106de None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:33.317660 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279838e-e443-4e1e-98a8-c9470099ae3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:33.318529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279838e-e443-4e1e-98a8-c9470099ae3b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:33.319000 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279838e-e443-4e1e-98a8-c9470099ae3b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:33.319448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279838e-e443-4e1e-98a8-c9470099ae3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:33.348607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d47bc3-9d43-409f-8919-76216b83fc53 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:33.349134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d47bc3-9d43-409f-8919-76216b83fc53 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:33.349134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d47bc3-9d43-409f-8919-76216b83fc53 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:33.351241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 218/428] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:33 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:00:33.356615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d8c4e-884f-4141-970f-669e6626d5cf None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:33.356909 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d8c4e-884f-4141-970f-669e6626d5cf None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:33.357193 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d8c4e-884f-4141-970f-669e6626d5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:33.387961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4b9d8c4e-884f-4141-970f-669e6626d5cf None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:33.389276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4b9d8c4e-884f-4141-970f-669e6626d5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:33.406091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279838e-e443-4e1e-98a8-c9470099ae3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:33.409685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d279838e-e443-4e1e-98a8-c9470099ae3b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:33.415125 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:33.488968 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:33.506853 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 219/429] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:33.524305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d8c4e-884f-4141-970f-669e6626d5cf None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:33.599795 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 211/430] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:33 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:33.604331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279838e-e443-4e1e-98a8-c9470099ae3b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:33.776120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d8c4e-884f-4141-970f-669e6626d5cf None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:33.794962 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:33.812736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279838e-e443-4e1e-98a8-c9470099ae3b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:33.854610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfca7e-25b7-42b4-af39-f8e1705334ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:33.855308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfca7e-25b7-42b4-af39-f8e1705334ef None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:33.855596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfca7e-25b7-42b4-af39-f8e1705334ef None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:33.855936 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfca7e-25b7-42b4-af39-f8e1705334ef None admin] PATH_INFO: `/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:33.857147 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfca7e-25b7-42b4-af39-f8e1705334ef None admin] RBAC: Authorizing `identity:get_project(project_id=5b8cad7e0f1a4ca3bb57873952d8ed80)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:33.858744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfca7e-25b7-42b4-af39-f8e1705334ef None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:33.865923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 212/431] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:33 2022] GET /identity/v3/projects/5b8cad7e0f1a4ca3bb57873952d8ed80 => generated 299 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:33.867137 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfca7e-25b7-42b4-af39-f8e1705334ef None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:33.868480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670edf2-dfeb-4002-b820-1e0fd37e9063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:33.870700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670edf2-dfeb-4002-b820-1e0fd37e9063 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:33.872822 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670edf2-dfeb-4002-b820-1e0fd37e9063 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:33.873876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670edf2-dfeb-4002-b820-1e0fd37e9063 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:33.912299 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670edf2-dfeb-4002-b820-1e0fd37e9063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-1670edf2-dfeb-4002-b820-1e0fd37e9063 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:33.925037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:33.925954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:33.926771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:33.926771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:33.926771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:33.926771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:33.926771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:33.926771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:33.926771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:33.926771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:33.926771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:33.926771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:33.926771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 15:00:33.926771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:33.932633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 220/432] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:33.948949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670edf2-dfeb-4002-b820-1e0fd37e9063 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:34.012349 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfca7e-25b7-42b4-af39-f8e1705334ef None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:34.061975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42343d7d-737e-40c9-9068-0e1626686885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:34.062621 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42343d7d-737e-40c9-9068-0e1626686885 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:34.062920 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42343d7d-737e-40c9-9068-0e1626686885 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:34.063198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42343d7d-737e-40c9-9068-0e1626686885 None admin] PATH_INFO: `/v3/groups/67c25d20e3e34c0b9d74a852083437c0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:34.064380 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42343d7d-737e-40c9-9068-0e1626686885 None admin] RBAC: Authorizing `identity:get_group(group_id=67c25d20e3e34c0b9d74a852083437c0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:34.065888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42343d7d-737e-40c9-9068-0e1626686885 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:34.067698 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 213/433] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:33 2022] GET /identity/v3/groups/67c25d20e3e34c0b9d74a852083437c0 => generated 227 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:34.073727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8f33e-6019-4841-9dcb-19cb3a03f371 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:34.074016 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8f33e-6019-4841-9dcb-19cb3a03f371 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:34.074292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8f33e-6019-4841-9dcb-19cb3a03f371 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:34.075658 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 214/434] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:34 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:34.076365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8f33e-6019-4841-9dcb-19cb3a03f371 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:34.102219 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:34.107759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:34.158360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670edf2-dfeb-4002-b820-1e0fd37e9063 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:34.197793 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6774466-4e71-44ea-892a-013a5cc2ca37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:34.198428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6774466-4e71-44ea-892a-013a5cc2ca37 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:34.198800 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6774466-4e71-44ea-892a-013a5cc2ca37 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:34.199081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6774466-4e71-44ea-892a-013a5cc2ca37 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:34.199808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6774466-4e71-44ea-892a-013a5cc2ca37 None admin] RBAC: Adding query filter params (name=member) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:34.200117 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6774466-4e71-44ea-892a-013a5cc2ca37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:34.201507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6774466-4e71-44ea-892a-013a5cc2ca37 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:34.208899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8f33e-6019-4841-9dcb-19cb3a03f371 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:34.217186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 221/435] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:00:33 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:34.218569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8936afa-e1f7-4eda-9144-347b5368735a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:34.218957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8936afa-e1f7-4eda-9144-347b5368735a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:34.219249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8936afa-e1f7-4eda-9144-347b5368735a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:34.241105 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c8936afa-e1f7-4eda-9144-347b5368735a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:34.242313 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c8936afa-e1f7-4eda-9144-347b5368735a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:34.252473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bec443-3785-4646-8233-69677f457de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:34.253093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bec443-3785-4646-8233-69677f457de7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:34.253378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bec443-3785-4646-8233-69677f457de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:34.253651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bec443-3785-4646-8233-69677f457de7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:34.254450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bec443-3785-4646-8233-69677f457de7 None admin] RBAC: Adding query filter params (group.id=67c25d20e3e34c0b9d74a852083437c0, role.id=d1f6973ce11a4644b649b38f8b2df173, scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:34.254803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bec443-3785-4646-8233-69677f457de7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:34.256251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bec443-3785-4646-8233-69677f457de7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:34.262203 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 215/436] 10.222.0.100 () {56 vars in 1381 bytes} [Mon Aug 1 15:00:34 2022] GET /identity/v3/role_assignments?group.id=67c25d20e3e34c0b9d74a852083437c0&scope.project.id=5b8cad7e0f1a4ca3bb57873952d8ed80&role.id=d1f6973ce11a4644b649b38f8b2df173 => generated 613 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:34.263023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bec443-3785-4646-8233-69677f457de7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:34.349441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 222/437] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:34.351569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4e2f5-dc5a-4053-8074-07986d8a42f2 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:34.352097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4e2f5-dc5a-4053-8074-07986d8a42f2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:34.352599 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4e2f5-dc5a-4053-8074-07986d8a42f2 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:34.354541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 223/438] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:34 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:34.365099 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f70b0-c7d4-48b8-983a-3b719503afa2 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:34.365759 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f70b0-c7d4-48b8-983a-3b719503afa2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:34.366321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f70b0-c7d4-48b8-983a-3b719503afa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:34.370927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:34.388160 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-fb4f70b0-c7d4-48b8-983a-3b719503afa2 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:34.389173 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bec443-3785-4646-8233-69677f457de7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:34.389400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fb4f70b0-c7d4-48b8-983a-3b719503afa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:34.434017 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1377b-bab0-4d8e-aa44-943a31d1b2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:34.434655 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1377b-bab0-4d8e-aa44-943a31d1b2eb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:34.434966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1377b-bab0-4d8e-aa44-943a31d1b2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:34.435247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1377b-bab0-4d8e-aa44-943a31d1b2eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:34.447146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:34.478057 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 224/439] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:34.478994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f70b0-c7d4-48b8-983a-3b719503afa2 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:34.536268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1377b-bab0-4d8e-aa44-943a31d1b2eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:34.538869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1377b-bab0-4d8e-aa44-943a31d1b2eb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:34.611589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f70b0-c7d4-48b8-983a-3b719503afa2 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:34.645622 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:34.656130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f1a11-39c9-4082-91d6-564bd2b53d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:34.656405 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f1a11-39c9-4082-91d6-564bd2b53d6a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:34.656549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f1a11-39c9-4082-91d6-564bd2b53d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:34.656820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283f1a11-39c9-4082-91d6-564bd2b53d6a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:34.666817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 216/440] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:34 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:34.667664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1377b-bab0-4d8e-aa44-943a31d1b2eb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:34.682245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283f1a11-39c9-4082-91d6-564bd2b53d6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-283f1a11-39c9-4082-91d6-564bd2b53d6a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:34.687407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:34.688342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:34.694606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:34.694606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:34.694606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:34.694606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:34.694606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:34.694606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:34.694606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:34.694606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:34.694606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:34.694606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:34.694606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:34.694606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:34.694606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 225/441] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:00:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:34.694606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f1a11-39c9-4082-91d6-564bd2b53d6a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:34.753295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:34.797861 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1377b-bab0-4d8e-aa44-943a31d1b2eb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:34.830394 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283f1a11-39c9-4082-91d6-564bd2b53d6a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:34.838225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9e71b-7572-4761-8f43-5c62b2d6b4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:34.838895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9e71b-7572-4761-8f43-5c62b2d6b4bd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:34.839175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9e71b-7572-4761-8f43-5c62b2d6b4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:34.839448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9e71b-7572-4761-8f43-5c62b2d6b4bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:34.849972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9e71b-7572-4761-8f43-5c62b2d6b4bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4ce9e71b-7572-4761-8f43-5c62b2d6b4bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:34.854929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:34.855859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:34.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:34.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:34.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:34.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:34.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:34.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:34.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:34.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:34.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:34.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:34.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:00:34.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:34.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 217/442] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:00:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:34.857529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9e71b-7572-4761-8f43-5c62b2d6b4bd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:34.869831 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f62b0-1cbc-4c24-ac18-62aee1023df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:34.870481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f62b0-1cbc-4c24-ac18-62aee1023df3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:34.870803 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f62b0-1cbc-4c24-ac18-62aee1023df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:34.871084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f62b0-1cbc-4c24-ac18-62aee1023df3 None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:34.872294 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f62b0-1cbc-4c24-ac18-62aee1023df3 None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:34.873754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f62b0-1cbc-4c24-ac18-62aee1023df3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:34.875494 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 226/443] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:34 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:34.876272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f62b0-1cbc-4c24-ac18-62aee1023df3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:34.919781 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:34.981242 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9e71b-7572-4761-8f43-5c62b2d6b4bd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:34.983035 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f62b0-1cbc-4c24-ac18-62aee1023df3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:34.994625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:35.018353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb62db-e23f-41ac-9835-2bfce3ee1d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:35.019018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb62db-e23f-41ac-9835-2bfce3ee1d82 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:35.019374 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb62db-e23f-41ac-9835-2bfce3ee1d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:35.019585 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb62db-e23f-41ac-9835-2bfce3ee1d82 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:35.020307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb62db-e23f-41ac-9835-2bfce3ee1d82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:35.020619 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb62db-e23f-41ac-9835-2bfce3ee1d82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:35.020936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9665a-eb6d-42e1-98ce-8c1747cb9195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:35.021566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9665a-eb6d-42e1-98ce-8c1747cb9195 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:35.021866 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9665a-eb6d-42e1-98ce-8c1747cb9195 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:35.022215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9665a-eb6d-42e1-98ce-8c1747cb9195 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:35.022307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb62db-e23f-41ac-9835-2bfce3ee1d82 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:35.023004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9665a-eb6d-42e1-98ce-8c1747cb9195 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:35.023310 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9665a-eb6d-42e1-98ce-8c1747cb9195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:35.024691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9665a-eb6d-42e1-98ce-8c1747cb9195 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:35.031073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 218/444] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:34 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:35.031923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb62db-e23f-41ac-9835-2bfce3ee1d82 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:35.033893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 227/445] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:34 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:35.036697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9665a-eb6d-42e1-98ce-8c1747cb9195 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:35.142854 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9665a-eb6d-42e1-98ce-8c1747cb9195 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:35.149539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb62db-e23f-41ac-9835-2bfce3ee1d82 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:35.158947 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:35.178358 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb22020-9bba-4d20-af14-c364c8a62f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:35.179540 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb22020-9bba-4d20-af14-c364c8a62f1a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:35.179827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb22020-9bba-4d20-af14-c364c8a62f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:35.180646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb22020-9bba-4d20-af14-c364c8a62f1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:35.189416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c73b97-1602-45b2-b7f2-8a7e82efa397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:35.190081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c73b97-1602-45b2-b7f2-8a7e82efa397 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:35.190435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c73b97-1602-45b2-b7f2-8a7e82efa397 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:35.190644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c73b97-1602-45b2-b7f2-8a7e82efa397 None admin] PATH_INFO: `/v3/groups/67c25d20e3e34c0b9d74a852083437c0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:35.191844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c73b97-1602-45b2-b7f2-8a7e82efa397 None admin] RBAC: Authorizing `identity:get_group(group_id=67c25d20e3e34c0b9d74a852083437c0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:35.193299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c73b97-1602-45b2-b7f2-8a7e82efa397 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:35.194967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 219/446] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:35 2022] GET /identity/v3/groups/67c25d20e3e34c0b9d74a852083437c0 => generated 227 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:35.195728 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c73b97-1602-45b2-b7f2-8a7e82efa397 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:35.221051 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:35.254940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb22020-9bba-4d20-af14-c364c8a62f1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:35.256408 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb22020-9bba-4d20-af14-c364c8a62f1a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:35.309992 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c73b97-1602-45b2-b7f2-8a7e82efa397 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:35.346087 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db43710-e16e-46bf-8b48-9c4f212eef16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:35.346748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db43710-e16e-46bf-8b48-9c4f212eef16 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:35.347038 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db43710-e16e-46bf-8b48-9c4f212eef16 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:35.347317 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db43710-e16e-46bf-8b48-9c4f212eef16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:35.362457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 228/447] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:35 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:35.364340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb22020-9bba-4d20-af14-c364c8a62f1a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:35.399438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:35.423993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db43710-e16e-46bf-8b48-9c4f212eef16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:35.426055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db43710-e16e-46bf-8b48-9c4f212eef16 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:35.438739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:35.475179 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb22020-9bba-4d20-af14-c364c8a62f1a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:35.520131 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b74fa-4269-4109-bb46-3fac71cfd6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:35.520752 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b74fa-4269-4109-bb46-3fac71cfd6ef None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:35.521030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b74fa-4269-4109-bb46-3fac71cfd6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:35.521302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b74fa-4269-4109-bb46-3fac71cfd6ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:35.522126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b74fa-4269-4109-bb46-3fac71cfd6ef None admin] RBAC: Adding query filter params (group.id=67c25d20e3e34c0b9d74a852083437c0, role.id=d1f6973ce11a4644b649b38f8b2df173, scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:35.522433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b74fa-4269-4109-bb46-3fac71cfd6ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:35.523916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b74fa-4269-4109-bb46-3fac71cfd6ef None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:35.531246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 229/448] 10.222.0.100 () {56 vars in 1381 bytes} [Mon Aug 1 15:00:35 2022] GET /identity/v3/role_assignments?group.id=67c25d20e3e34c0b9d74a852083437c0&scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f&role.id=d1f6973ce11a4644b649b38f8b2df173 => generated 613 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:35.532074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b74fa-4269-4109-bb46-3fac71cfd6ef None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:35.554959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 220/449] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:35 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:35.560988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db43710-e16e-46bf-8b48-9c4f212eef16 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:35.658389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:35.661378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b74fa-4269-4109-bb46-3fac71cfd6ef None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:35.689668 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db43710-e16e-46bf-8b48-9c4f212eef16 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:35.705137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95449c03-94a0-4571-b7c0-7a02e0e47fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:35.705808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95449c03-94a0-4571-b7c0-7a02e0e47fcd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:35.706065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95449c03-94a0-4571-b7c0-7a02e0e47fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:35.706348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95449c03-94a0-4571-b7c0-7a02e0e47fcd None admin] PATH_INFO: `/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:35.707544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95449c03-94a0-4571-b7c0-7a02e0e47fcd None admin] RBAC: Authorizing `identity:get_project(project_id=ebb7ad89c9734ecbbd41c0c2de27069f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:35.709011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95449c03-94a0-4571-b7c0-7a02e0e47fcd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:35.710717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 230/450] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:35 2022] GET /identity/v3/projects/ebb7ad89c9734ecbbd41c0c2de27069f => generated 303 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:35.716574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95449c03-94a0-4571-b7c0-7a02e0e47fcd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:35.720686 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:35.742823 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506c5d9-6d99-4abf-b873-873d9882745e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:35.744225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506c5d9-6d99-4abf-b873-873d9882745e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:35.744524 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506c5d9-6d99-4abf-b873-873d9882745e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:35.744809 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506c5d9-6d99-4abf-b873-873d9882745e None admin] PATH_INFO: `/v3/projects/bcc1ff25500841a3a7180e5fff03ff9f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:35.746043 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506c5d9-6d99-4abf-b873-873d9882745e None admin] RBAC: Authorizing `identity:get_project(project_id=bcc1ff25500841a3a7180e5fff03ff9f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:35.747559 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506c5d9-6d99-4abf-b873-873d9882745e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:35.749291 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 221/451] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:00:35 2022] GET /identity/v3/projects/bcc1ff25500841a3a7180e5fff03ff9f => generated 345 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:35.755776 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506c5d9-6d99-4abf-b873-873d9882745e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:35.848994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95449c03-94a0-4571-b7c0-7a02e0e47fcd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:35.898023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506c5d9-6d99-4abf-b873-873d9882745e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:35.908110 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1571a-91b9-44be-8da5-3e1b02e00e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:35.909250 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1571a-91b9-44be-8da5-3e1b02e00e35 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:35.909765 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1571a-91b9-44be-8da5-3e1b02e00e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:35.910275 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1571a-91b9-44be-8da5-3e1b02e00e35 None admin] PATH_INFO: `/v3/groups/67c25d20e3e34c0b9d74a852083437c0` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:35.911964 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1571a-91b9-44be-8da5-3e1b02e00e35 None admin] RBAC: Authorizing `identity:get_group(group_id=67c25d20e3e34c0b9d74a852083437c0)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:35.916594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1571a-91b9-44be-8da5-3e1b02e00e35 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:35.918736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 231/452] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:35 2022] GET /identity/v3/groups/67c25d20e3e34c0b9d74a852083437c0 => generated 227 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:35.926114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1571a-91b9-44be-8da5-3e1b02e00e35 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:35.946409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d10bc-661d-4399-8762-d400bd1408da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:35.947867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d10bc-661d-4399-8762-d400bd1408da None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:35.948589 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d10bc-661d-4399-8762-d400bd1408da None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:35.949127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d10bc-661d-4399-8762-d400bd1408da None admin] PATH_INFO: `/v3/groups/17547fbfbfc2476e9eda70a0eedcd2a0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:35.950579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d10bc-661d-4399-8762-d400bd1408da None admin] RBAC: Authorizing `identity:get_group(group_id=17547fbfbfc2476e9eda70a0eedcd2a0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:35.952502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d10bc-661d-4399-8762-d400bd1408da None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:35.954629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 222/453] 10.222.0.100 () {56 vars in 1185 bytes} [Mon Aug 1 15:00:35 2022] GET /identity/v3/groups/17547fbfbfc2476e9eda70a0eedcd2a0 => generated 230 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:35.960345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d10bc-661d-4399-8762-d400bd1408da None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:35.977602 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:36.007245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:36.039439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1571a-91b9-44be-8da5-3e1b02e00e35 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:36.066859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d10bc-661d-4399-8762-d400bd1408da None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:36.074138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46864b89-3bc5-4cb7-8e52-fcda448cb4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:36.074725 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46864b89-3bc5-4cb7-8e52-fcda448cb4f4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:36.075011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46864b89-3bc5-4cb7-8e52-fcda448cb4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:36.075282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46864b89-3bc5-4cb7-8e52-fcda448cb4f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:36.076077 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46864b89-3bc5-4cb7-8e52-fcda448cb4f4 None admin] RBAC: Adding query filter params (group.id=67c25d20e3e34c0b9d74a852083437c0, role.id=57221d717a454c7fae4f2df1de9dbe5b, scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:36.076385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46864b89-3bc5-4cb7-8e52-fcda448cb4f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:36.077974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46864b89-3bc5-4cb7-8e52-fcda448cb4f4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:36.083971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 232/454] 10.222.0.100 () {56 vars in 1381 bytes} [Mon Aug 1 15:00:35 2022] GET /identity/v3/role_assignments?group.id=67c25d20e3e34c0b9d74a852083437c0&scope.project.id=ebb7ad89c9734ecbbd41c0c2de27069f&role.id=57221d717a454c7fae4f2df1de9dbe5b => generated 613 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:36.105770 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd8a90-8dc5-498b-b278-fc886e0233ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:36.106738 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd8a90-8dc5-498b-b278-fc886e0233ba None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:36.107096 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd8a90-8dc5-498b-b278-fc886e0233ba None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:36.107513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd8a90-8dc5-498b-b278-fc886e0233ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:36.108664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd8a90-8dc5-498b-b278-fc886e0233ba None admin] RBAC: Adding query filter params (group.id=17547fbfbfc2476e9eda70a0eedcd2a0, role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=bcc1ff25500841a3a7180e5fff03ff9f) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:36.109093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd8a90-8dc5-498b-b278-fc886e0233ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:36.111241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd8a90-8dc5-498b-b278-fc886e0233ba None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:36.126774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 223/455] 10.222.0.100 () {56 vars in 1381 bytes} [Mon Aug 1 15:00:35 2022] GET /identity/v3/role_assignments?group.id=17547fbfbfc2476e9eda70a0eedcd2a0&scope.project.id=bcc1ff25500841a3a7180e5fff03ff9f&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 613 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:38.753729 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8ac1-e35c-4565-ae77-59c2718c9bf5 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:38.754782 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8ac1-e35c-4565-ae77-59c2718c9bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:38.755287 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d8ac1-e35c-4565-ae77-59c2718c9bf5 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:38.756936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 233/456] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:38 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:38.762625 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260e0f4-1a89-4428-b47b-9ba631b8a73d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:38.763128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260e0f4-1a89-4428-b47b-9ba631b8a73d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:38.763272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260e0f4-1a89-4428-b47b-9ba631b8a73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:38.791684 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7260e0f4-1a89-4428-b47b-9ba631b8a73d None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:38.792906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7260e0f4-1a89-4428-b47b-9ba631b8a73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:38.893316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 224/457] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:38.937016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d8ac1-e35c-4565-ae77-59c2718c9bf5 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:39.057467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ab80d-b8c0-427a-82a6-81f09f590ab4 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:39.058051 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ab80d-b8c0-427a-82a6-81f09f590ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:39.058478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ab80d-b8c0-427a-82a6-81f09f590ab4 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:39.060351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 225/458] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:39 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:39.065342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95331f-cc2c-42f8-a70a-d390f5fa31ea None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:39.065843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95331f-cc2c-42f8-a70a-d390f5fa31ea None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:39.066268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d95331f-cc2c-42f8-a70a-d390f5fa31ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:39.087380 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:39.095794 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d8ac1-e35c-4565-ae77-59c2718c9bf5 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:39.108763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9d95331f-cc2c-42f8-a70a-d390f5fa31ea None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:39.110054 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9d95331f-cc2c-42f8-a70a-d390f5fa31ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:39.170395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e41beb-ddd0-4d11-b84a-f029a176a4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:39.171949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e41beb-ddd0-4d11-b84a-f029a176a4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:39.172235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e41beb-ddd0-4d11-b84a-f029a176a4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:39.172513 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e41beb-ddd0-4d11-b84a-f029a176a4c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:39.176768 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:39.189175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e41beb-ddd0-4d11-b84a-f029a176a4c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:39.190747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e41beb-ddd0-4d11-b84a-f029a176a4c8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-d5e41beb-ddd0-4d11-b84a-f029a176a4c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:39.217074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:39.218030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:39.218894 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:39.218894 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:39.218894 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:39.218894 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:39.224451 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 234/459] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:39.227684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e41beb-ddd0-4d11-b84a-f029a176a4c8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:39.288718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 226/460] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:39.326274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95331f-cc2c-42f8-a70a-d390f5fa31ea None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:39.382221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e41beb-ddd0-4d11-b84a-f029a176a4c8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:39.417943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5f224-29df-48fb-9c1d-703d24af3533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:39.418996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5f224-29df-48fb-9c1d-703d24af3533 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:39.419478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5f224-29df-48fb-9c1d-703d24af3533 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:39.419947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5f224-29df-48fb-9c1d-703d24af3533 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:39.420891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5f224-29df-48fb-9c1d-703d24af3533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:39.421406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5f224-29df-48fb-9c1d-703d24af3533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:39.423021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5f224-29df-48fb-9c1d-703d24af3533 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:39.445030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 235/461] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:39.446919 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b9f75-5cf3-459f-95a4-5877bcdd36e7 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:39.447215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b9f75-5cf3-459f-95a4-5877bcdd36e7 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:39.447493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b9f75-5cf3-459f-95a4-5877bcdd36e7 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:39.449304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 236/462] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:39 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:39.450417 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b9f75-5cf3-459f-95a4-5877bcdd36e7 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:39.468708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d95331f-cc2c-42f8-a70a-d390f5fa31ea None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:39.509647 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:39.516323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce764ffc-1dee-4893-bd5f-a5ced2ab5138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:39.517041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce764ffc-1dee-4893-bd5f-a5ced2ab5138 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:39.517327 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce764ffc-1dee-4893-bd5f-a5ced2ab5138 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:39.517623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce764ffc-1dee-4893-bd5f-a5ced2ab5138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:39.520575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:39.541767 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce764ffc-1dee-4893-bd5f-a5ced2ab5138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:39.543301 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce764ffc-1dee-4893-bd5f-a5ced2ab5138 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-ce764ffc-1dee-4893-bd5f-a5ced2ab5138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:39.561863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:39.563413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:39.564933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:39.564933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:39.564933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:39.564933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:39.564933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 227/463] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:39.564933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d97bad-9111-4b1d-8a68-818bcc04cdeb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:39.565246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d97bad-9111-4b1d-8a68-818bcc04cdeb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:39.565728 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d97bad-9111-4b1d-8a68-818bcc04cdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:39.580176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b9f75-5cf3-459f-95a4-5877bcdd36e7 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:39.586588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d3d97bad-9111-4b1d-8a68-818bcc04cdeb None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:39.588057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d3d97bad-9111-4b1d-8a68-818bcc04cdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:39.618239 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15d82a-01bb-44e3-84ca-f1d84569bcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:39.618896 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15d82a-01bb-44e3-84ca-f1d84569bcd4 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:39.619180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15d82a-01bb-44e3-84ca-f1d84569bcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:39.619485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15d82a-01bb-44e3-84ca-f1d84569bcd4 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:39.620182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15d82a-01bb-44e3-84ca-f1d84569bcd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:39.621627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15d82a-01bb-44e3-84ca-f1d84569bcd4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:39.656549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 237/464] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:00:39 2022] POST /identity/v3/users => generated 247 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:00:39.659646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22514b9-2428-4582-9469-c7a7db6f3dfa None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:39.660124 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22514b9-2428-4582-9469-c7a7db6f3dfa None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:39.664724 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22514b9-2428-4582-9469-c7a7db6f3dfa None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:39.668719 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 238/465] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:39 2022] GET /identity => generated 271 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:39.669891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22514b9-2428-4582-9469-c7a7db6f3dfa None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:39.688908 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 228/466] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:39.691432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77a210-1709-42b5-960d-ba74f5ecd8b0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:39.692225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77a210-1709-42b5-960d-ba74f5ecd8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:39.692739 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77a210-1709-42b5-960d-ba74f5ecd8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:39.716248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ec77a210-1709-42b5-960d-ba74f5ecd8b0 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:39.718052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ec77a210-1709-42b5-960d-ba74f5ecd8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:39.806584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22514b9-2428-4582-9469-c7a7db6f3dfa None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:39.825089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 229/467] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:39.826609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77a210-1709-42b5-960d-ba74f5ecd8b0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:39.826997 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:39.856756 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27811e2d-cac5-4033-b46c-908a2f10dc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:39.857389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27811e2d-cac5-4033-b46c-908a2f10dc58 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:39.858414 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27811e2d-cac5-4033-b46c-908a2f10dc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:39.858414 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27811e2d-cac5-4033-b46c-908a2f10dc58 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:39.858704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27811e2d-cac5-4033-b46c-908a2f10dc58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:39.858973 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27811e2d-cac5-4033-b46c-908a2f10dc58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:39.860492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27811e2d-cac5-4033-b46c-908a2f10dc58 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:39.875755 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 239/468] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:39.877027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27811e2d-cac5-4033-b46c-908a2f10dc58 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:39.892850 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:39.966213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77a210-1709-42b5-960d-ba74f5ecd8b0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:40.014249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27811e2d-cac5-4033-b46c-908a2f10dc58 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:40.024463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1efbc-656e-42d4-82a7-da1342073feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:40.025658 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1efbc-656e-42d4-82a7-da1342073feb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:40.027241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1efbc-656e-42d4-82a7-da1342073feb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:40.028566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1efbc-656e-42d4-82a7-da1342073feb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:40.041122 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1efbc-656e-42d4-82a7-da1342073feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:40.042746 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1efbc-656e-42d4-82a7-da1342073feb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-80a1efbc-656e-42d4-82a7-da1342073feb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:40.060045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:40.061055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:40.063253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:40.063253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:40.063253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:40.063253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:40.063253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 230/469] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:40.063253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1efbc-656e-42d4-82a7-da1342073feb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:40.069827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca98dc4-adc0-443c-bc1a-c50b9cec9f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:40.070493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca98dc4-adc0-443c-bc1a-c50b9cec9f35 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:40.070811 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca98dc4-adc0-443c-bc1a-c50b9cec9f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:40.071094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca98dc4-adc0-443c-bc1a-c50b9cec9f35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:40.088219 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca98dc4-adc0-443c-bc1a-c50b9cec9f35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:40.090292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca98dc4-adc0-443c-bc1a-c50b9cec9f35 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-eca98dc4-adc0-443c-bc1a-c50b9cec9f35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:40.111074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:40.111983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:40.112823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:40.112823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:40.112823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:40.112823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:40.115229 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 240/470] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:40.116709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca98dc4-adc0-443c-bc1a-c50b9cec9f35 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:40.140723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:40.210063 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:40.213384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1efbc-656e-42d4-82a7-da1342073feb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:40.237863 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca98dc4-adc0-443c-bc1a-c50b9cec9f35 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:40.251954 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41853d-3842-497d-85dc-2e73e15dc741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:40.252590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41853d-3842-497d-85dc-2e73e15dc741 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:40.252875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41853d-3842-497d-85dc-2e73e15dc741 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:40.253154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41853d-3842-497d-85dc-2e73e15dc741 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:40.253850 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41853d-3842-497d-85dc-2e73e15dc741 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:40.255337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41853d-3842-497d-85dc-2e73e15dc741 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:40.276138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbbcb7-ed1f-4bd4-89c9-cf7d8dd8489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:40.276771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbbcb7-ed1f-4bd4-89c9-cf7d8dd8489d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:40.277062 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbbcb7-ed1f-4bd4-89c9-cf7d8dd8489d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:40.277326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbbcb7-ed1f-4bd4-89c9-cf7d8dd8489d None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:40.278025 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbbcb7-ed1f-4bd4-89c9-cf7d8dd8489d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:40.278324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbbcb7-ed1f-4bd4-89c9-cf7d8dd8489d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:40.279591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 231/471] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:00:40 2022] POST /identity/v3/users => generated 249 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:00:40.279849 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbbcb7-ed1f-4bd4-89c9-cf7d8dd8489d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:40.280618 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41853d-3842-497d-85dc-2e73e15dc741 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:40.307243 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 241/472] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:40.309706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafcac6-9e3d-431c-b06d-c10827430f0d None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:40.310303 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafcac6-9e3d-431c-b06d-c10827430f0d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:40.310857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafcac6-9e3d-431c-b06d-c10827430f0d None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:40.314164 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 242/473] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:40 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:40.315500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafcac6-9e3d-431c-b06d-c10827430f0d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:40.417680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41853d-3842-497d-85dc-2e73e15dc741 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:40.429132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:40.466300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafcac6-9e3d-431c-b06d-c10827430f0d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:40.471339 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128466ff-7e81-48bb-817a-e0ff034f2d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:40.471974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128466ff-7e81-48bb-817a-e0ff034f2d05 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:40.472256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128466ff-7e81-48bb-817a-e0ff034f2d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:40.472531 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128466ff-7e81-48bb-817a-e0ff034f2d05 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:40.473234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128466ff-7e81-48bb-817a-e0ff034f2d05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:40.473538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128466ff-7e81-48bb-817a-e0ff034f2d05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:40.475035 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128466ff-7e81-48bb-817a-e0ff034f2d05 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:40.492189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 232/474] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:40.493398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6ff77-9ade-4351-8190-51eb7161559c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:40.493684 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6ff77-9ade-4351-8190-51eb7161559c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:40.494024 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb6ff77-9ade-4351-8190-51eb7161559c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:40.509531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddac473-02aa-43a0-be58-02d7c2b227b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:40.510457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddac473-02aa-43a0-be58-02d7c2b227b2 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:40.511300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddac473-02aa-43a0-be58-02d7c2b227b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:40.511300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddac473-02aa-43a0-be58-02d7c2b227b2 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:40.511784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddac473-02aa-43a0-be58-02d7c2b227b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:40.513355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddac473-02aa-43a0-be58-02d7c2b227b2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:40.514114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ceb6ff77-9ade-4351-8190-51eb7161559c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:40.515908 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ceb6ff77-9ade-4351-8190-51eb7161559c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:40.529734 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:40.540327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 243/475] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:00:40 2022] POST /identity/v3/users => generated 249 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:00:40.543987 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddac473-02aa-43a0-be58-02d7c2b227b2 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:40.614096 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 233/476] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:40.641195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6ff77-9ade-4351-8190-51eb7161559c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:40.665675 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddac473-02aa-43a0-be58-02d7c2b227b2 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:40.698999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:40.706117 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4474d44-51f1-46c7-941f-050dcde2a944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:40.707030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4474d44-51f1-46c7-941f-050dcde2a944 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:40.707457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4474d44-51f1-46c7-941f-050dcde2a944 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:40.707902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4474d44-51f1-46c7-941f-050dcde2a944 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:40.708730 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4474d44-51f1-46c7-941f-050dcde2a944 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:40.710951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4474d44-51f1-46c7-941f-050dcde2a944 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:40.745160 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 244/477] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:00:40 2022] POST /identity/v3/users => generated 250 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:00:40.783354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb6ff77-9ade-4351-8190-51eb7161559c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:40.829636 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cacb0-cf33-4e39-b52d-b70bf7b2cfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:40.830300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cacb0-cf33-4e39-b52d-b70bf7b2cfba None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:40.830551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cacb0-cf33-4e39-b52d-b70bf7b2cfba None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:40.830841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4cacb0-cf33-4e39-b52d-b70bf7b2cfba None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:40.831578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cacb0-cf33-4e39-b52d-b70bf7b2cfba None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:40.832974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4cacb0-cf33-4e39-b52d-b70bf7b2cfba None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:40.843500 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.sql.core [None req-cd4cacb0-cf33-4e39-b52d-b70bf7b2cfba None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 15:00:40.843500 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 15:00:40.843500 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [parameters: {'id': '674b20956164401791019bee90f57bac', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 15:00:40.843500 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80358) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cd4cacb0-cf33-4e39-b52d-b70bf7b2cfba None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result.read() Aug 01 15:00:40.861801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return ref.to_dict() Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:00:40.862820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application yield resource Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise self.value Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application session.commit() Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 15:00:40.863744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.session.flush() Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application compat.raise_( Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 15:00:40.864609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:00:40.865497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application result.read() Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:00:40.867965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [parameters: {'id': '674b20956164401791019bee90f57bac', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:40.869012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 15:00:40.870008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:40.871126 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 234/478] 10.222.0.100 () {60 vars in 1136 bytes} [Mon Aug 1 15:00:40 2022] POST /identity/v3/roles => generated 144 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:40.872305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4474d44-51f1-46c7-941f-050dcde2a944 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:41.015675 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4474d44-51f1-46c7-941f-050dcde2a944 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:41.066567 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490e89f-3847-4e14-850f-716491bdce33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:41.068201 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490e89f-3847-4e14-850f-716491bdce33 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:41.069523 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490e89f-3847-4e14-850f-716491bdce33 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:41.070285 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490e89f-3847-4e14-850f-716491bdce33 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:41.085507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490e89f-3847-4e14-850f-716491bdce33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-9490e89f-3847-4e14-850f-716491bdce33 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:41.094410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:41.095524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:41.096341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:41.096341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:41.096341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:41.096341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:41.096341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:41.096341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:41.096341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:41.096341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:41.096341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:41.096341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:41.096341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 15:00:41.096341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:41.100039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 245/479] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:00:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:41.106369 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cacb0-cf33-4e39-b52d-b70bf7b2cfba None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:41.123067 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:41.221286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4cacb0-cf33-4e39-b52d-b70bf7b2cfba None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:41.260241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a64b32-bfa6-4464-82ad-cd5bf922e207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:41.260879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a64b32-bfa6-4464-82ad-cd5bf922e207 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:41.261158 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a64b32-bfa6-4464-82ad-cd5bf922e207 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:41.261518 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a64b32-bfa6-4464-82ad-cd5bf922e207 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:41.262248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a64b32-bfa6-4464-82ad-cd5bf922e207 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:41.262552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a64b32-bfa6-4464-82ad-cd5bf922e207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:41.263985 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a64b32-bfa6-4464-82ad-cd5bf922e207 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:41.273799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 235/480] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:41 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:42.200855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cac24-de89-4dc5-9714-1aba9337ee74 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:42.201409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cac24-de89-4dc5-9714-1aba9337ee74 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:42.201409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cac24-de89-4dc5-9714-1aba9337ee74 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:42.203007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 246/481] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:42.208207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87433a4-6efa-4f8f-aeb5-0339864572cb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:42.208612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87433a4-6efa-4f8f-aeb5-0339864572cb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:42.208829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87433a4-6efa-4f8f-aeb5-0339864572cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:42.235670 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f87433a4-6efa-4f8f-aeb5-0339864572cb None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:42.236867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f87433a4-6efa-4f8f-aeb5-0339864572cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:42.327225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 236/482] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:42 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:42.363629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cac24-de89-4dc5-9714-1aba9337ee74 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:42.439821 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:42.514066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cac24-de89-4dc5-9714-1aba9337ee74 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:42.557352 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62899c-2114-43a6-a3af-d0d99ee77133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:42.558186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62899c-2114-43a6-a3af-d0d99ee77133 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:42.558630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62899c-2114-43a6-a3af-d0d99ee77133 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:42.559081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62899c-2114-43a6-a3af-d0d99ee77133 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:42.571019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb62899c-2114-43a6-a3af-d0d99ee77133 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-cb62899c-2114-43a6-a3af-d0d99ee77133 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:42.576199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:42.577427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:42.578424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:42.578424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:42.578424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:42.578424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:42.578424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:42.578424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:42.578424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:42.578424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:42.578424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:42.578424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:42.578424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:42.578424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:42.580615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 247/483] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:42.586595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87433a4-6efa-4f8f-aeb5-0339864572cb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:42.589456 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:42.717539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87433a4-6efa-4f8f-aeb5-0339864572cb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:42.765171 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d3533-1a20-4876-a4cf-64b88dba6c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:42.767735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d3533-1a20-4876-a4cf-64b88dba6c92 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:42.768221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d3533-1a20-4876-a4cf-64b88dba6c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:42.768639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d3533-1a20-4876-a4cf-64b88dba6c92 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:42.769526 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d3533-1a20-4876-a4cf-64b88dba6c92 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:42.769974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d3533-1a20-4876-a4cf-64b88dba6c92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:42.771787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d3533-1a20-4876-a4cf-64b88dba6c92 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:42.789843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 237/484] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:00:42 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:42.795842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62899c-2114-43a6-a3af-d0d99ee77133 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:42.865046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4299f-f365-4771-80f6-8cb384372bbd None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:42.865330 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4299f-f365-4771-80f6-8cb384372bbd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:42.865605 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4299f-f365-4771-80f6-8cb384372bbd None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:42.867883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 238/485] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:42 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:42.875041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89065492-d35d-4e2a-bdef-03a260fe29b3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:42.875326 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89065492-d35d-4e2a-bdef-03a260fe29b3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:42.875603 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89065492-d35d-4e2a-bdef-03a260fe29b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:42.896459 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-89065492-d35d-4e2a-bdef-03a260fe29b3 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:42.898821 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-89065492-d35d-4e2a-bdef-03a260fe29b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:42.980209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:42.982704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 239/486] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:42 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:00:43.009555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89065492-d35d-4e2a-bdef-03a260fe29b3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:43.011254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62899c-2114-43a6-a3af-d0d99ee77133 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:43.025811 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:43.104799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4c1b2-ca39-4ba4-9b0b-fb0715f41842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:43.105834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4c1b2-ca39-4ba4-9b0b-fb0715f41842 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:43.106315 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4c1b2-ca39-4ba4-9b0b-fb0715f41842 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:43.106817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4c1b2-ca39-4ba4-9b0b-fb0715f41842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:43.133182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4c1b2-ca39-4ba4-9b0b-fb0715f41842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:43.135040 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed4c1b2-ca39-4ba4-9b0b-fb0715f41842 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:43.135970 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89065492-d35d-4e2a-bdef-03a260fe29b3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-eed4c1b2-ca39-4ba4-9b0b-fb0715f41842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:43.165028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:43.165946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:43.166839 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:43.166839 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:43.166839 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:43.166839 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:43.173182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 248/487] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:43.179006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4c1b2-ca39-4ba4-9b0b-fb0715f41842 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:43.185280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba00833-120b-47a8-8df9-afe874a1ac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:43.185946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba00833-120b-47a8-8df9-afe874a1ac58 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:43.186224 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba00833-120b-47a8-8df9-afe874a1ac58 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:43.186768 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba00833-120b-47a8-8df9-afe874a1ac58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:43.211618 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba00833-120b-47a8-8df9-afe874a1ac58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-dba00833-120b-47a8-8df9-afe874a1ac58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:43.217706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:43.218615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:43.219475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:43.219475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:43.219475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:43.219475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:43.219475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:43.219475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:43.219475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:43.219475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:43.219475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:43.219475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:43.219475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:43.219475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:43.228457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 240/488] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:43.236136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba00833-120b-47a8-8df9-afe874a1ac58 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:43.304003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4c1b2-ca39-4ba4-9b0b-fb0715f41842 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:43.345523 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1362c6a-171e-449d-8b2b-6a7fbbf91dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:43.346338 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1362c6a-171e-449d-8b2b-6a7fbbf91dbe None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:43.347098 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1362c6a-171e-449d-8b2b-6a7fbbf91dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:43.347485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1362c6a-171e-449d-8b2b-6a7fbbf91dbe None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:43.348455 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1362c6a-171e-449d-8b2b-6a7fbbf91dbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:43.348988 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1362c6a-171e-449d-8b2b-6a7fbbf91dbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:43.350898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1362c6a-171e-449d-8b2b-6a7fbbf91dbe None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:43.370691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 249/489] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:43.373947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1362c6a-171e-449d-8b2b-6a7fbbf91dbe None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:43.419034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:43.419223 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:43.479599 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba00833-120b-47a8-8df9-afe874a1ac58 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:43.517131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c92f9-d6cb-478e-854b-53e9f357a5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:43.517864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c92f9-d6cb-478e-854b-53e9f357a5bf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:43.518217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c92f9-d6cb-478e-854b-53e9f357a5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:43.518557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c92f9-d6cb-478e-854b-53e9f357a5bf None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:43.519314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c92f9-d6cb-478e-854b-53e9f357a5bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:43.519616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c92f9-d6cb-478e-854b-53e9f357a5bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:43.521073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c92f9-d6cb-478e-854b-53e9f357a5bf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:43.531067 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 241/490] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:00:43 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:43.532559 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726af61-c945-47b4-80fd-280c702e987a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:43.532840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726af61-c945-47b4-80fd-280c702e987a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:43.533164 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6726af61-c945-47b4-80fd-280c702e987a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:43.539304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1362c6a-171e-449d-8b2b-6a7fbbf91dbe None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:43.547547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 242/491] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:43.548957 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff17cd0-1dba-4cc8-b6cd-16b87c134a07 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:43.549247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff17cd0-1dba-4cc8-b6cd-16b87c134a07 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:43.549530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff17cd0-1dba-4cc8-b6cd-16b87c134a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:43.569896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7ff17cd0-1dba-4cc8-b6cd-16b87c134a07 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:43.571064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7ff17cd0-1dba-4cc8-b6cd-16b87c134a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:43.589115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eab1e5-9846-45a8-adcb-10110a057f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:43.589764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eab1e5-9846-45a8-adcb-10110a057f8d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:43.590039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eab1e5-9846-45a8-adcb-10110a057f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:43.590315 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eab1e5-9846-45a8-adcb-10110a057f8d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:43.598365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eab1e5-9846-45a8-adcb-10110a057f8d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:43.599916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eab1e5-9846-45a8-adcb-10110a057f8d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-92eab1e5-9846-45a8-adcb-10110a057f8d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:00:43.614262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:43.615485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:43.617381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:43.617381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:43.617381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:00:43.617381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:00:43.617381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:00:43.617381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:00:43.617381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:00:43.617381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:00:43.617381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:00:43.617381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:43.620698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 250/492] 10.222.0.100 () {56 vars in 1098 bytes} [Mon Aug 1 15:00:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:43.621768 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eab1e5-9846-45a8-adcb-10110a057f8d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:43.650897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 243/493] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:43.650897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101dfbc-f193-4bf9-a694-a04dd63311a6 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:43.650897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101dfbc-f193-4bf9-a694-a04dd63311a6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:43.650897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d101dfbc-f193-4bf9-a694-a04dd63311a6 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:43.655047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 244/494] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:43 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:43.655946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101dfbc-f193-4bf9-a694-a04dd63311a6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:43.708214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:43.720864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:43.744935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eab1e5-9846-45a8-adcb-10110a057f8d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:43.766333 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d101dfbc-f193-4bf9-a694-a04dd63311a6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:43.784346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08903fa9-e9da-4437-a0ce-e56e5e49338f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:43.785255 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08903fa9-e9da-4437-a0ce-e56e5e49338f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:43.785487 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08903fa9-e9da-4437-a0ce-e56e5e49338f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:43.785828 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08903fa9-e9da-4437-a0ce-e56e5e49338f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:43.799058 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08903fa9-e9da-4437-a0ce-e56e5e49338f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:43.800665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08903fa9-e9da-4437-a0ce-e56e5e49338f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:43.808116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecc788-e37d-4193-9660-dcd7ae9fa955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:43.808824 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc788-e37d-4193-9660-dcd7ae9fa955 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:43.809175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc788-e37d-4193-9660-dcd7ae9fa955 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:43.809512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecc788-e37d-4193-9660-dcd7ae9fa955 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:43.810381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecc788-e37d-4193-9660-dcd7ae9fa955 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:43.810828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecc788-e37d-4193-9660-dcd7ae9fa955 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:43.812488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecc788-e37d-4193-9660-dcd7ae9fa955 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-08903fa9-e9da-4437-a0ce-e56e5e49338f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:43.816617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:43.817491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:43.818341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:43.818341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:43.818341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:43.818341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:43.818341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 251/495] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:43.819420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddd474-5d1b-4df4-8cb2-cfb44a8e0d7b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:43.819703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddd474-5d1b-4df4-8cb2-cfb44a8e0d7b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:43.820029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddd474-5d1b-4df4-8cb2-cfb44a8e0d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:43.839605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4dddd474-5d1b-4df4-8cb2-cfb44a8e0d7b None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:43.840876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4dddd474-5d1b-4df4-8cb2-cfb44a8e0d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:43.858076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 245/496] 10.222.0.100 () {56 vars in 1138 bytes} [Mon Aug 1 15:00:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:43.859317 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecc788-e37d-4193-9660-dcd7ae9fa955 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:43.920437 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 252/497] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:43.921758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddd474-5d1b-4df4-8cb2-cfb44a8e0d7b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:43.973807 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecc788-e37d-4193-9660-dcd7ae9fa955 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:43.976734 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:43.996602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:44.013011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141bfcf-1fdc-40a5-b164-a10f158d2c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:44.013659 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141bfcf-1fdc-40a5-b164-a10f158d2c63 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:44.013939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141bfcf-1fdc-40a5-b164-a10f158d2c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:44.014212 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141bfcf-1fdc-40a5-b164-a10f158d2c63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:44.025416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141bfcf-1fdc-40a5-b164-a10f158d2c63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4141bfcf-1fdc-40a5-b164-a10f158d2c63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:44.030421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:44.031507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:44.032476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:44.032476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:44.032476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:44.032476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:44.032476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:44.032476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:44.032476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:44.032476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:44.032476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:44.032476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:44.032476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:44.032476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:44.032476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 246/498] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:44.033255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141bfcf-1fdc-40a5-b164-a10f158d2c63 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:44.045239 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddd474-5d1b-4df4-8cb2-cfb44a8e0d7b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:44.081193 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e59fb-c7cb-4522-aa22-472127febd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:44.081737 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e59fb-c7cb-4522-aa22-472127febd8b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:44.082026 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e59fb-c7cb-4522-aa22-472127febd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:44.082302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e59fb-c7cb-4522-aa22-472127febd8b None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:44.083029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e59fb-c7cb-4522-aa22-472127febd8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:44.083337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e59fb-c7cb-4522-aa22-472127febd8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:44.085314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e59fb-c7cb-4522-aa22-472127febd8b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:44.096955 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 253/499] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:44.097764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e59fb-c7cb-4522-aa22-472127febd8b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:44.142167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141bfcf-1fdc-40a5-b164-a10f158d2c63 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:44.175697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27baeb7e-e5fb-4c90-b978-1c746e6526bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:44.176309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27baeb7e-e5fb-4c90-b978-1c746e6526bb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:44.176590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27baeb7e-e5fb-4c90-b978-1c746e6526bb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:44.176864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27baeb7e-e5fb-4c90-b978-1c746e6526bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:44.195665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e59fb-c7cb-4522-aa22-472127febd8b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:44.207994 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:44.216273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:44.231022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41642728-87c4-4ea7-ac53-b20e7b6c2109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:44.231663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41642728-87c4-4ea7-ac53-b20e7b6c2109 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:44.231949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41642728-87c4-4ea7-ac53-b20e7b6c2109 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:44.232230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41642728-87c4-4ea7-ac53-b20e7b6c2109 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:44.242318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41642728-87c4-4ea7-ac53-b20e7b6c2109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:44.244843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27baeb7e-e5fb-4c90-b978-1c746e6526bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:44.246305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27baeb7e-e5fb-4c90-b978-1c746e6526bb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-41642728-87c4-4ea7-ac53-b20e7b6c2109 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:44.248498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:44.249605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:44.250415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:44.250415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:44.250415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:44.250415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:44.250415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:44.250415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:44.250415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:44.250415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:44.250415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:44.250415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:44.250415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:44.250415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:44.250415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 254/500] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:44.251432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e8ea3-e245-4ab1-9f3a-3a37cb12ebca None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:44.251432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e8ea3-e245-4ab1-9f3a-3a37cb12ebca None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:44.251432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e8ea3-e245-4ab1-9f3a-3a37cb12ebca None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:44.252712 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 255/501] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:44 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:44.253266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e8ea3-e245-4ab1-9f3a-3a37cb12ebca None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:44.348247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 247/502] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:44 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:44.349022 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27baeb7e-e5fb-4c90-b978-1c746e6526bb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:44.354980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e8ea3-e245-4ab1-9f3a-3a37cb12ebca None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:44.398865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eee24-5a1a-46de-997c-1fa0a3604bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:44.399490 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eee24-5a1a-46de-997c-1fa0a3604bde None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:44.399771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eee24-5a1a-46de-997c-1fa0a3604bde None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:44.400046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eee24-5a1a-46de-997c-1fa0a3604bde None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:44.400768 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eee24-5a1a-46de-997c-1fa0a3604bde None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:44.401153 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eee24-5a1a-46de-997c-1fa0a3604bde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:44.402560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eee24-5a1a-46de-997c-1fa0a3604bde None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:44.411725 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 256/503] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:00:44 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:44.412603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eee24-5a1a-46de-997c-1fa0a3604bde None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:44.435035 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:44.469800 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:44.485486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27baeb7e-e5fb-4c90-b978-1c746e6526bb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:44.524308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a35d14-7a8a-4221-bbc7-d56a3d9715e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:44.524959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a35d14-7a8a-4221-bbc7-d56a3d9715e0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:44.525244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a35d14-7a8a-4221-bbc7-d56a3d9715e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:44.525519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a35d14-7a8a-4221-bbc7-d56a3d9715e0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:44.533573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a35d14-7a8a-4221-bbc7-d56a3d9715e0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:44.535086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a35d14-7a8a-4221-bbc7-d56a3d9715e0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:44.541397 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eee24-5a1a-46de-997c-1fa0a3604bde None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-01a35d14-7a8a-4221-bbc7-d56a3d9715e0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:00:44.550749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:44.551707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:44.552596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:44.552596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:44.552596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:00:44.552596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:00:44.552596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:00:44.552596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:00:44.552596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:00:44.552596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:00:44.552596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:00:44.552596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:44.552596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 248/504] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:44.553817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46afa5b-1916-423b-8e96-b4eb10214d3f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:44.554089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46afa5b-1916-423b-8e96-b4eb10214d3f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:44.554318 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46afa5b-1916-423b-8e96-b4eb10214d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:44.574713 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a46afa5b-1916-423b-8e96-b4eb10214d3f None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:44.576224 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a46afa5b-1916-423b-8e96-b4eb10214d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:44.595917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20c3ae-0c86-401a-b091-cc3a81349ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:44.596908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20c3ae-0c86-401a-b091-cc3a81349ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:44.597366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20c3ae-0c86-401a-b091-cc3a81349ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:44.597816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20c3ae-0c86-401a-b091-cc3a81349ec7 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:44.599362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20c3ae-0c86-401a-b091-cc3a81349ec7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:44.599807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20c3ae-0c86-401a-b091-cc3a81349ec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:44.602200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b20c3ae-0c86-401a-b091-cc3a81349ec7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:44.613842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 257/505] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:00:44 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:44.614683 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20c3ae-0c86-401a-b091-cc3a81349ec7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:44.646999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 249/506] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:44.647451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46afa5b-1916-423b-8e96-b4eb10214d3f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:44.705973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:44.729814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b20c3ae-0c86-401a-b091-cc3a81349ec7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:44.732495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:44.754932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46afa5b-1916-423b-8e96-b4eb10214d3f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:44.767709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96223082-5d67-4638-8cb8-27d41fd6418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:44.768345 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96223082-5d67-4638-8cb8-27d41fd6418a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:44.768640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96223082-5d67-4638-8cb8-27d41fd6418a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:44.768915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96223082-5d67-4638-8cb8-27d41fd6418a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:44.780054 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96223082-5d67-4638-8cb8-27d41fd6418a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:44.781516 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96223082-5d67-4638-8cb8-27d41fd6418a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:44.796721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003db18-ad3e-434e-a7b3-5fbf5f80f273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:44.797562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003db18-ad3e-434e-a7b3-5fbf5f80f273 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:44.798009 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003db18-ad3e-434e-a7b3-5fbf5f80f273 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:44.798441 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003db18-ad3e-434e-a7b3-5fbf5f80f273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-96223082-5d67-4638-8cb8-27d41fd6418a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:44.804885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:44.805836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:44.806724 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:44.806724 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:44.806724 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:44.806724 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:44.806724 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 258/507] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:44.806984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96223082-5d67-4638-8cb8-27d41fd6418a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:44.811780 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003db18-ad3e-434e-a7b3-5fbf5f80f273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:44.813279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003db18-ad3e-434e-a7b3-5fbf5f80f273 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3003db18-ad3e-434e-a7b3-5fbf5f80f273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:44.829815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:44.830840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:44.832656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:44.832656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:44.832656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:44.832656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:44.832656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 250/508] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:44.832656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003db18-ad3e-434e-a7b3-5fbf5f80f273 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:44.920153 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96223082-5d67-4638-8cb8-27d41fd6418a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:44.935404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003db18-ad3e-434e-a7b3-5fbf5f80f273 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:44.942506 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:44.956092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53fda9-36aa-447f-bb23-334f1d156efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:44.956860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53fda9-36aa-447f-bb23-334f1d156efe None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:44.957251 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53fda9-36aa-447f-bb23-334f1d156efe None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:44.957639 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53fda9-36aa-447f-bb23-334f1d156efe None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:44.958538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53fda9-36aa-447f-bb23-334f1d156efe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:44.958975 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53fda9-36aa-447f-bb23-334f1d156efe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:44.961114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53fda9-36aa-447f-bb23-334f1d156efe None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:44.971117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679043f-285e-4dda-ad62-9166f635db1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:44.971767 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679043f-285e-4dda-ad62-9166f635db1f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:44.972054 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679043f-285e-4dda-ad62-9166f635db1f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:44.972332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679043f-285e-4dda-ad62-9166f635db1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:44.982967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679043f-285e-4dda-ad62-9166f635db1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:44.984476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679043f-285e-4dda-ad62-9166f635db1f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:44.998474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 259/509] 10.222.0.100 () {56 vars in 1142 bytes} [Mon Aug 1 15:00:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2679043f-285e-4dda-ad62-9166f635db1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:44.998803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:44.999643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:45.000494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:45.000494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:45.000494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:45.000494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:45.000703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53fda9-36aa-447f-bb23-334f1d156efe None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:45.000771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 251/510] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:45.000836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679043f-285e-4dda-ad62-9166f635db1f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:45.018333 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:45.105956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53fda9-36aa-447f-bb23-334f1d156efe None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:45.108034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679043f-285e-4dda-ad62-9166f635db1f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:45.140984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d857da-098b-4382-97f4-6fe620167d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:45.141633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d857da-098b-4382-97f4-6fe620167d3d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:45.141918 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d857da-098b-4382-97f4-6fe620167d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:45.142012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6b92a-5d7a-419c-ad27-744081e83e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:45.142197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d857da-098b-4382-97f4-6fe620167d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:45.142592 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6b92a-5d7a-419c-ad27-744081e83e2c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:45.142915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6b92a-5d7a-419c-ad27-744081e83e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:45.143193 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6b92a-5d7a-419c-ad27-744081e83e2c None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:45.143889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b6b92a-5d7a-419c-ad27-744081e83e2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:45.144200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b6b92a-5d7a-419c-ad27-744081e83e2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:45.145585 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b6b92a-5d7a-419c-ad27-744081e83e2c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:45.153198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d857da-098b-4382-97f4-6fe620167d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:45.154694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d857da-098b-4382-97f4-6fe620167d3d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:45.158610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 252/511] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:45.159468 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6b92a-5d7a-419c-ad27-744081e83e2c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-72d857da-098b-4382-97f4-6fe620167d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:45.170594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:45.171638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:45.172474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:45.172474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:45.172474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:45.172474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:45.172474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 260/512] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:45.172474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d857da-098b-4382-97f4-6fe620167d3d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:45.183718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:45.238440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:45.268024 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6b92a-5d7a-419c-ad27-744081e83e2c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:45.277087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d857da-098b-4382-97f4-6fe620167d3d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:45.302582 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b77e4-0e95-4e09-9a0d-9113bd559955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:45.303725 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b77e4-0e95-4e09-9a0d-9113bd559955 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:45.304007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b77e4-0e95-4e09-9a0d-9113bd559955 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:45.304277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b77e4-0e95-4e09-9a0d-9113bd559955 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:45.304964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b77e4-0e95-4e09-9a0d-9113bd559955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:45.305295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b77e4-0e95-4e09-9a0d-9113bd559955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:45.306691 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b77e4-0e95-4e09-9a0d-9113bd559955 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:45.311412 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e58a5-e7fc-49b1-83d5-e92375b10ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:45.312036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e58a5-e7fc-49b1-83d5-e92375b10ead None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:45.312325 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e58a5-e7fc-49b1-83d5-e92375b10ead None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:45.312605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e58a5-e7fc-49b1-83d5-e92375b10ead None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:45.318381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 253/513] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:45.319797 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b77e4-0e95-4e09-9a0d-9113bd559955 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:45.387270 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e58a5-e7fc-49b1-83d5-e92375b10ead None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:45.388742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e58a5-e7fc-49b1-83d5-e92375b10ead None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:45.406247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:45.431125 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b77e4-0e95-4e09-9a0d-9113bd559955 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:45.453977 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:45.467709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29521b04-98f1-4bf8-aa58-43eb96f6be60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:45.468334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29521b04-98f1-4bf8-aa58-43eb96f6be60 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:45.468614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29521b04-98f1-4bf8-aa58-43eb96f6be60 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:45.468887 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29521b04-98f1-4bf8-aa58-43eb96f6be60 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:45.469581 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29521b04-98f1-4bf8-aa58-43eb96f6be60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:45.469888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29521b04-98f1-4bf8-aa58-43eb96f6be60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:45.471301 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29521b04-98f1-4bf8-aa58-43eb96f6be60 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:45.486038 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 254/514] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:45.486695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29521b04-98f1-4bf8-aa58-43eb96f6be60 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:45.501217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 261/515] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:45 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:45.502064 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e58a5-e7fc-49b1-83d5-e92375b10ead None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:45.598982 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29521b04-98f1-4bf8-aa58-43eb96f6be60 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:45.612099 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e58a5-e7fc-49b1-83d5-e92375b10ead None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:45.642555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29459fa4-d6fb-4d07-80cc-0d73e7f7042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:45.643418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29459fa4-d6fb-4d07-80cc-0d73e7f7042a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:45.643796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29459fa4-d6fb-4d07-80cc-0d73e7f7042a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:45.644076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29459fa4-d6fb-4d07-80cc-0d73e7f7042a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:45.650258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b267b6-6727-45e2-a7fa-9946967a4c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:45.650892 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b267b6-6727-45e2-a7fa-9946967a4c94 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:45.651174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b267b6-6727-45e2-a7fa-9946967a4c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:45.651266 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29459fa4-d6fb-4d07-80cc-0d73e7f7042a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:45.651448 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b267b6-6727-45e2-a7fa-9946967a4c94 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:45.652136 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b267b6-6727-45e2-a7fa-9946967a4c94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:45.652441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b267b6-6727-45e2-a7fa-9946967a4c94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:45.652687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29459fa4-d6fb-4d07-80cc-0d73e7f7042a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:45.653872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b267b6-6727-45e2-a7fa-9946967a4c94 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:45.654458 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:45.666441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 262/516] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:45.667207 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b267b6-6727-45e2-a7fa-9946967a4c94 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-29459fa4-d6fb-4d07-80cc-0d73e7f7042a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:00:45.667604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:45.668651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:45.669709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:00:45.669709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:00:45.669709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:00:45.669709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:00:45.669709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:00:45.669709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:00:45.669709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:45.669709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:45.673246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 255/517] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:45.674112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29459fa4-d6fb-4d07-80cc-0d73e7f7042a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:45.710923 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:45.780118 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29459fa4-d6fb-4d07-80cc-0d73e7f7042a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:45.786589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b267b6-6727-45e2-a7fa-9946967a4c94 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:45.815997 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee7534-f946-4507-905c-9f9c7da4d5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:45.816647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee7534-f946-4507-905c-9f9c7da4d5fc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:45.816926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee7534-f946-4507-905c-9f9c7da4d5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:45.817207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee7534-f946-4507-905c-9f9c7da4d5fc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:45.823023 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735501af-9bcd-436b-8605-878a554c5bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:45.823675 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735501af-9bcd-436b-8605-878a554c5bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:45.823957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735501af-9bcd-436b-8605-878a554c5bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:45.824232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735501af-9bcd-436b-8605-878a554c5bf4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:45.825546 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee7534-f946-4507-905c-9f9c7da4d5fc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:45.827034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee7534-f946-4507-905c-9f9c7da4d5fc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:45.832923 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735501af-9bcd-436b-8605-878a554c5bf4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:45.834404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735501af-9bcd-436b-8605-878a554c5bf4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-80ee7534-f946-4507-905c-9f9c7da4d5fc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:00:45.839839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:45.840806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:45.841708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:45.841708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:45.841708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:00:45.841708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:00:45.841708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:00:45.841708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:00:45.841708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:00:45.841708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:00:45.841708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:00:45.841708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:45.841708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 256/518] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:45.842425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee7534-f946-4507-905c-9f9c7da4d5fc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-735501af-9bcd-436b-8605-878a554c5bf4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:00:45.851309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:45.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:45.853217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:45.853217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:45.853217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:00:45.853217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:00:45.853217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:00:45.853217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:00:45.853217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:00:45.853217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:00:45.853217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:00:45.853217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:45.853217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 263/519] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:45.854395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735501af-9bcd-436b-8605-878a554c5bf4 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:45.908547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:45.943155 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:45.953996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee7534-f946-4507-905c-9f9c7da4d5fc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:45.965342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735501af-9bcd-436b-8605-878a554c5bf4 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:45.988747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a68de-aa39-405c-a8d1-d3101a68d3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:45.989391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a68de-aa39-405c-a8d1-d3101a68d3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:45.989674 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a68de-aa39-405c-a8d1-d3101a68d3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:45.989950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a68de-aa39-405c-a8d1-d3101a68d3e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:46.000720 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8b382-560a-4463-bc10-f3630be95ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:46.000883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a68de-aa39-405c-a8d1-d3101a68d3e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:46.001337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8b382-560a-4463-bc10-f3630be95ba7 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:46.001621 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8b382-560a-4463-bc10-f3630be95ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:46.001898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8b382-560a-4463-bc10-f3630be95ba7 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:46.002334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a68de-aa39-405c-a8d1-d3101a68d3e9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:46.002575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8b382-560a-4463-bc10-f3630be95ba7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:46.004038 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8b382-560a-4463-bc10-f3630be95ba7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a14a68de-aa39-405c-a8d1-d3101a68d3e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:46.016810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:46.018001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:46.019221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:46.019221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:46.019221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:46.019221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:46.019221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 257/520] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:46.019221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a68de-aa39-405c-a8d1-d3101a68d3e9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:46.029706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 264/521] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:00:45 2022] POST /identity/v3/users => generated 248 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:00:46.030555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8b382-560a-4463-bc10-f3630be95ba7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:46.142654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a68de-aa39-405c-a8d1-d3101a68d3e9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:46.150517 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8b382-560a-4463-bc10-f3630be95ba7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:46.155645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:46.194906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d0a0c-46bb-4bc8-9114-69bb04c05a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:46.195715 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d0a0c-46bb-4bc8-9114-69bb04c05a09 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:46.196623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d0a0c-46bb-4bc8-9114-69bb04c05a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:46.196623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d0a0c-46bb-4bc8-9114-69bb04c05a09 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:46.197319 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d0a0c-46bb-4bc8-9114-69bb04c05a09 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:46.197709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d0a0c-46bb-4bc8-9114-69bb04c05a09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:46.199258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d0a0c-46bb-4bc8-9114-69bb04c05a09 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:46.208576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268b2e8-f6b1-45eb-820d-2a23e87f2f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:46.209348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268b2e8-f6b1-45eb-820d-2a23e87f2f5c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:46.209728 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268b2e8-f6b1-45eb-820d-2a23e87f2f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:46.210010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268b2e8-f6b1-45eb-820d-2a23e87f2f5c None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:46.211216 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268b2e8-f6b1-45eb-820d-2a23e87f2f5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:46.211216 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268b2e8-f6b1-45eb-820d-2a23e87f2f5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:46.212588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268b2e8-f6b1-45eb-820d-2a23e87f2f5c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:46.217327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:46.221672 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 265/522] 10.222.0.100 () {56 vars in 1144 bytes} [Mon Aug 1 15:00:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:46.222999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d0a0c-46bb-4bc8-9114-69bb04c05a09 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:46.238248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 258/523] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:00:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:46.239469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268b2e8-f6b1-45eb-820d-2a23e87f2f5c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:46.336694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d0a0c-46bb-4bc8-9114-69bb04c05a09 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:46.349540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268b2e8-f6b1-45eb-820d-2a23e87f2f5c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:46.374390 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09162de2-34e7-4409-bb93-1dd6c33f922e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:46.375340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09162de2-34e7-4409-bb93-1dd6c33f922e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:46.375844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09162de2-34e7-4409-bb93-1dd6c33f922e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:46.376188 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09162de2-34e7-4409-bb93-1dd6c33f922e None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:46.376989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09162de2-34e7-4409-bb93-1dd6c33f922e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:46.377301 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09162de2-34e7-4409-bb93-1dd6c33f922e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:46.378781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09162de2-34e7-4409-bb93-1dd6c33f922e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:46.388081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03bb0f-f9ac-482e-80da-5d735e61a22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:46.388704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03bb0f-f9ac-482e-80da-5d735e61a22e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:46.388983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03bb0f-f9ac-482e-80da-5d735e61a22e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:46.389258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03bb0f-f9ac-482e-80da-5d735e61a22e None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:46.389950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03bb0f-f9ac-482e-80da-5d735e61a22e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:46.390259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03bb0f-f9ac-482e-80da-5d735e61a22e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:46.391670 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03bb0f-f9ac-482e-80da-5d735e61a22e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:46.399265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 266/524] 10.222.0.100 () {56 vars in 1142 bytes} [Mon Aug 1 15:00:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:46.400697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09162de2-34e7-4409-bb93-1dd6c33f922e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:46.408288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 259/525] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:46.409322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03bb0f-f9ac-482e-80da-5d735e61a22e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:46.444245 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:46.467602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:46.520316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09162de2-34e7-4409-bb93-1dd6c33f922e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:46.527080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03bb0f-f9ac-482e-80da-5d735e61a22e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:46.563699 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01b195-b5b0-4fd2-8b14-73d4b6110f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:46.564321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01b195-b5b0-4fd2-8b14-73d4b6110f73 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:46.564605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01b195-b5b0-4fd2-8b14-73d4b6110f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:46.564880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01b195-b5b0-4fd2-8b14-73d4b6110f73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:46.566560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec716f-71a4-487f-b852-84bc30d0426d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:46.567206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec716f-71a4-487f-b852-84bc30d0426d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:46.567738 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec716f-71a4-487f-b852-84bc30d0426d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:46.568017 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec716f-71a4-487f-b852-84bc30d0426d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:46.569699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec716f-71a4-487f-b852-84bc30d0426d None admin] RBAC: Adding query filter params (role.id=70e0c42f1bdd49178db398637f021bbc, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=ec50ac7c54324884a1b2a6c63d617907) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:46.570012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec716f-71a4-487f-b852-84bc30d0426d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:46.571488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec716f-71a4-487f-b852-84bc30d0426d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:46.590208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 260/526] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:46 2022] GET /identity/v3/role_assignments?user.id=ec50ac7c54324884a1b2a6c63d617907&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=70e0c42f1bdd49178db398637f021bbc => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:46.590208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec716f-71a4-487f-b852-84bc30d0426d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:46.654762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01b195-b5b0-4fd2-8b14-73d4b6110f73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:46.656327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01b195-b5b0-4fd2-8b14-73d4b6110f73 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:46.725888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:46.731148 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:46.741145 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec716f-71a4-487f-b852-84bc30d0426d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:46.775155 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 267/527] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:46 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:46.776108 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01b195-b5b0-4fd2-8b14-73d4b6110f73 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:46.783813 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfdd30-4b08-4044-a9ef-0eff9b8b3501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:46.785227 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfdd30-4b08-4044-a9ef-0eff9b8b3501 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:46.785863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfdd30-4b08-4044-a9ef-0eff9b8b3501 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:46.786507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfdd30-4b08-4044-a9ef-0eff9b8b3501 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:46.884917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfdd30-4b08-4044-a9ef-0eff9b8b3501 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:46.886610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfdd30-4b08-4044-a9ef-0eff9b8b3501 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:46.890788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01b195-b5b0-4fd2-8b14-73d4b6110f73 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:46.927437 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449855a9-ba16-4d45-bb8e-c9f8a5a934ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:46.927691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449855a9-ba16-4d45-bb8e-c9f8a5a934ef None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:46.927852 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449855a9-ba16-4d45-bb8e-c9f8a5a934ef None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:46.928671 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449855a9-ba16-4d45-bb8e-c9f8a5a934ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:46.936353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449855a9-ba16-4d45-bb8e-c9f8a5a934ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:46.937931 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449855a9-ba16-4d45-bb8e-c9f8a5a934ef None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-449855a9-ba16-4d45-bb8e-c9f8a5a934ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:00:46.951475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:46.952655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:46.953859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:00:46.953859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:00:46.953859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:00:46.953859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:00:46.953859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:00:46.953859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:00:46.953859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:46.953859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:46.953859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 268/528] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:46.954366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449855a9-ba16-4d45-bb8e-c9f8a5a934ef None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:46.975974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:47.002278 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:47.013213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 261/529] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:46 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:47.014303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfdd30-4b08-4044-a9ef-0eff9b8b3501 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:47.071038 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449855a9-ba16-4d45-bb8e-c9f8a5a934ef None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:47.108384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c4c8e-65d2-477d-9228-6d69ac44908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:47.109097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c4c8e-65d2-477d-9228-6d69ac44908e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:47.109385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c4c8e-65d2-477d-9228-6d69ac44908e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:47.109662 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c4c8e-65d2-477d-9228-6d69ac44908e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:47.121082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c4c8e-65d2-477d-9228-6d69ac44908e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:47.122610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c4c8e-65d2-477d-9228-6d69ac44908e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f97c4c8e-65d2-477d-9228-6d69ac44908e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:47.140430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:47.141764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:47.142979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:47.142979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:47.142979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:47.142979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:47.143201 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfdd30-4b08-4044-a9ef-0eff9b8b3501 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:47.143274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 269/530] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:47.143274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c4c8e-65d2-477d-9228-6d69ac44908e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:47.179980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f2fcc-ef19-4eef-8b6c-4a4977da61ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:47.180720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f2fcc-ef19-4eef-8b6c-4a4977da61ee None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:47.181069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f2fcc-ef19-4eef-8b6c-4a4977da61ee None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:47.181413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13f2fcc-ef19-4eef-8b6c-4a4977da61ee None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:47.182339 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f2fcc-ef19-4eef-8b6c-4a4977da61ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:47.182822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f2fcc-ef19-4eef-8b6c-4a4977da61ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:47.184461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13f2fcc-ef19-4eef-8b6c-4a4977da61ee None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:47.197664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 262/531] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:00:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:47.198793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f2fcc-ef19-4eef-8b6c-4a4977da61ee None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:47.228295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:47.266937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:47.276311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c4c8e-65d2-477d-9228-6d69ac44908e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:47.321138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13f2fcc-ef19-4eef-8b6c-4a4977da61ee None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:47.325602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a522cfa-78bc-400a-b33c-e8bb2cdd5bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:47.326332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a522cfa-78bc-400a-b33c-e8bb2cdd5bfd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:47.326709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a522cfa-78bc-400a-b33c-e8bb2cdd5bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:47.327068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a522cfa-78bc-400a-b33c-e8bb2cdd5bfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:47.342630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a522cfa-78bc-400a-b33c-e8bb2cdd5bfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:47.344860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a522cfa-78bc-400a-b33c-e8bb2cdd5bfd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:47.360302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419a715-e7a9-4c8d-9718-b90c2ad41efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:47.361572 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419a715-e7a9-4c8d-9718-b90c2ad41efa None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:47.361999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419a715-e7a9-4c8d-9718-b90c2ad41efa None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:47.362448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419a715-e7a9-4c8d-9718-b90c2ad41efa None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:47.363329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419a715-e7a9-4c8d-9718-b90c2ad41efa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:47.363827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419a715-e7a9-4c8d-9718-b90c2ad41efa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2a522cfa-78bc-400a-b33c-e8bb2cdd5bfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:47.364610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:47.365650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:47.367246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:47.367246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:47.367246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:47.367246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:47.368380 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419a715-e7a9-4c8d-9718-b90c2ad41efa None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:47.369010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 270/532] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:47.371632 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a522cfa-78bc-400a-b33c-e8bb2cdd5bfd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:47.388142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 263/533] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:47.389231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419a715-e7a9-4c8d-9718-b90c2ad41efa None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:47.503567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419a715-e7a9-4c8d-9718-b90c2ad41efa None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:47.520100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:47.522679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a522cfa-78bc-400a-b33c-e8bb2cdd5bfd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:47.555654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:47.564295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf0d9f-d202-4ce8-a483-cb9982d83aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:47.565276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf0d9f-d202-4ce8-a483-cb9982d83aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:47.566150 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf0d9f-d202-4ce8-a483-cb9982d83aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:47.567043 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf0d9f-d202-4ce8-a483-cb9982d83aa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:47.568005 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf0d9f-d202-4ce8-a483-cb9982d83aa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:47.568495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf0d9f-d202-4ce8-a483-cb9982d83aa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:47.569664 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ee331-17aa-4763-85b5-60d8bf1ada54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:47.570492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ee331-17aa-4763-85b5-60d8bf1ada54 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:47.570741 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf0d9f-d202-4ce8-a483-cb9982d83aa7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:47.571197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ee331-17aa-4763-85b5-60d8bf1ada54 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:47.571648 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ee331-17aa-4763-85b5-60d8bf1ada54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:47.572976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ee331-17aa-4763-85b5-60d8bf1ada54 None admin] RBAC: Adding query filter params (role.id=70e0c42f1bdd49178db398637f021bbc, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=ead1b36d35f3485f89edd15b8443feea) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:47.573473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ee331-17aa-4763-85b5-60d8bf1ada54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:47.575726 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ee331-17aa-4763-85b5-60d8bf1ada54 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:47.583785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 271/534] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:47 2022] GET /identity/v3/role_assignments?user.id=ead1b36d35f3485f89edd15b8443feea&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=70e0c42f1bdd49178db398637f021bbc => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:47.591376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 264/535] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:47.591580 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ee331-17aa-4763-85b5-60d8bf1ada54 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:47.592926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf0d9f-d202-4ce8-a483-cb9982d83aa7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:47.810138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf0d9f-d202-4ce8-a483-cb9982d83aa7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:47.862104 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c78cc0-6aca-4cd6-b067-ad7032063e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:47.863286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c78cc0-6aca-4cd6-b067-ad7032063e5d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:47.863778 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c78cc0-6aca-4cd6-b067-ad7032063e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:47.864279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c78cc0-6aca-4cd6-b067-ad7032063e5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:47.864689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ee331-17aa-4763-85b5-60d8bf1ada54 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:47.874362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c78cc0-6aca-4cd6-b067-ad7032063e5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:47.876298 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c78cc0-6aca-4cd6-b067-ad7032063e5d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-10c78cc0-6aca-4cd6-b067-ad7032063e5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:00:47.900070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:47.900996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:47.901866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:00:47.901866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:00:47.901866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:00:47.901866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:00:47.901866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:00:47.901866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:00:47.901866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:47.901866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:47.901866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 265/536] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:47.909005 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c78cc0-6aca-4cd6-b067-ad7032063e5d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:47.963886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56495be7-e8e8-4060-a4e3-450f66cf4f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:47.964552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56495be7-e8e8-4060-a4e3-450f66cf4f7a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:47.964832 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56495be7-e8e8-4060-a4e3-450f66cf4f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:47.965103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56495be7-e8e8-4060-a4e3-450f66cf4f7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:47.971109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:47.987632 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56495be7-e8e8-4060-a4e3-450f66cf4f7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:47.989122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56495be7-e8e8-4060-a4e3-450f66cf4f7a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-56495be7-e8e8-4060-a4e3-450f66cf4f7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:00:48.020045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:48.020993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:48.021908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:00:48.021908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:00:48.021908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:00:48.021908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:00:48.021908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:00:48.021908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:00:48.021908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:48.021908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:48.021908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 272/537] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:48.028162 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56495be7-e8e8-4060-a4e3-450f66cf4f7a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:48.065124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c78cc0-6aca-4cd6-b067-ad7032063e5d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:48.098016 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062a991-f7d1-426f-b008-99e10acd5879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:48.098728 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062a991-f7d1-426f-b008-99e10acd5879 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:48.099169 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062a991-f7d1-426f-b008-99e10acd5879 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:48.099454 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062a991-f7d1-426f-b008-99e10acd5879 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:48.100759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062a991-f7d1-426f-b008-99e10acd5879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:48.101075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062a991-f7d1-426f-b008-99e10acd5879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:48.102530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062a991-f7d1-426f-b008-99e10acd5879 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:48.117733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 266/538] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:00:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:48.131950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062a991-f7d1-426f-b008-99e10acd5879 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:48.171172 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:48.282326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56495be7-e8e8-4060-a4e3-450f66cf4f7a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:48.330864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee49c93-c65c-4191-8400-df84b5a388f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:48.331855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee49c93-c65c-4191-8400-df84b5a388f5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:48.332343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee49c93-c65c-4191-8400-df84b5a388f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:48.333624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee49c93-c65c-4191-8400-df84b5a388f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:48.335152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee49c93-c65c-4191-8400-df84b5a388f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:48.338081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee49c93-c65c-4191-8400-df84b5a388f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:48.345489 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee49c93-c65c-4191-8400-df84b5a388f5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:48.361918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062a991-f7d1-426f-b008-99e10acd5879 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:48.367018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:48.379202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 273/539] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:00:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:48.380941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fdf01-6949-419a-8edf-7e2298683d45 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:48.384493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fdf01-6949-419a-8edf-7e2298683d45 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:48.384793 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fdf01-6949-419a-8edf-7e2298683d45 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:48.386318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 274/540] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:48 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:48.390742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fdf01-6949-419a-8edf-7e2298683d45 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:48.421111 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa256dc-a714-4ffa-b126-7066ec8b5430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:48.422537 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa256dc-a714-4ffa-b126-7066ec8b5430 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:48.423323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa256dc-a714-4ffa-b126-7066ec8b5430 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:48.423803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa256dc-a714-4ffa-b126-7066ec8b5430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:48.425215 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa256dc-a714-4ffa-b126-7066ec8b5430 None admin] RBAC: Adding query filter params (role.id=70e0c42f1bdd49178db398637f021bbc, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=e75edfc5bb404d63b27a355d5ab8af01) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:48.426016 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa256dc-a714-4ffa-b126-7066ec8b5430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:48.427746 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa256dc-a714-4ffa-b126-7066ec8b5430 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:48.442724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 267/541] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:48 2022] GET /identity/v3/role_assignments?user.id=e75edfc5bb404d63b27a355d5ab8af01&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=70e0c42f1bdd49178db398637f021bbc => generated 260 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:48.442724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc700e-d464-4bb8-913b-7d6c8754c948 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:48.442724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc700e-d464-4bb8-913b-7d6c8754c948 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:48.442724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc700e-d464-4bb8-913b-7d6c8754c948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:48.489748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cebc700e-d464-4bb8-913b-7d6c8754c948 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:48.491737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cebc700e-d464-4bb8-913b-7d6c8754c948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:48.622128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fdf01-6949-419a-8edf-7e2298683d45 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:48.633672 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:48.653707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 268/542] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:48 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:48.673066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fbef4-be84-47fc-a810-9545be2463cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:48.673692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fbef4-be84-47fc-a810-9545be2463cd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:48.673974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fbef4-be84-47fc-a810-9545be2463cd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:48.674251 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fbef4-be84-47fc-a810-9545be2463cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:48.675104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fbef4-be84-47fc-a810-9545be2463cd None admin] RBAC: Adding query filter params (role.id=70e0c42f1bdd49178db398637f021bbc, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=e7f757eabcb84de6ab042e0ac5a8158c) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:48.675474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fbef4-be84-47fc-a810-9545be2463cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:48.676997 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fbef4-be84-47fc-a810-9545be2463cd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:48.684203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 275/543] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:48 2022] GET /identity/v3/role_assignments?user.id=e7f757eabcb84de6ab042e0ac5a8158c&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=70e0c42f1bdd49178db398637f021bbc => generated 260 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:48.686049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc700e-d464-4bb8-913b-7d6c8754c948 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:48.804273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:48.824890 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc700e-d464-4bb8-913b-7d6c8754c948 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:48.886048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225390a4-e7ef-4617-8d97-19ec1ccc12b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:48.886641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225390a4-e7ef-4617-8d97-19ec1ccc12b1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:48.886950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225390a4-e7ef-4617-8d97-19ec1ccc12b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:48.887223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225390a4-e7ef-4617-8d97-19ec1ccc12b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:48.901525 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225390a4-e7ef-4617-8d97-19ec1ccc12b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-225390a4-e7ef-4617-8d97-19ec1ccc12b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:48.906628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:48.907544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:48.908903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:48.908903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:48.908903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:48.908903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:48.908903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:48.908903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:48.908903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:48.908903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:48.908903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:48.908903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:48.908903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:48.908903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:48.908903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 269/544] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:48.914285 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fbef4-be84-47fc-a810-9545be2463cd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:48.996114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563eade-47c4-4071-9ec1-b7950db37751 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:48.996423 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563eade-47c4-4071-9ec1-b7950db37751 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:48.996679 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563eade-47c4-4071-9ec1-b7950db37751 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:48.998291 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 270/545] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:49.004700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b670853-6559-4276-98a7-d66f93c78d2f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:49.004977 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b670853-6559-4276-98a7-d66f93c78d2f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:49.005249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b670853-6559-4276-98a7-d66f93c78d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:49.034724 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fbef4-be84-47fc-a810-9545be2463cd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:49.035268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2b670853-6559-4276-98a7-d66f93c78d2f None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:49.036544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2b670853-6559-4276-98a7-d66f93c78d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:49.071600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a111ada-8f70-4914-9f77-eb5b98e6a43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:49.072228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a111ada-8f70-4914-9f77-eb5b98e6a43a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:49.072511 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a111ada-8f70-4914-9f77-eb5b98e6a43a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:49.072790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a111ada-8f70-4914-9f77-eb5b98e6a43a None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:49.073506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a111ada-8f70-4914-9f77-eb5b98e6a43a None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:49.073810 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a111ada-8f70-4914-9f77-eb5b98e6a43a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:49.075233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a111ada-8f70-4914-9f77-eb5b98e6a43a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:49.085079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 276/546] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:00:48 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:49.091090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a111ada-8f70-4914-9f77-eb5b98e6a43a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:49.128413 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:49.142977 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 271/547] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:00:49.161321 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b670853-6559-4276-98a7-d66f93c78d2f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:49.232082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a111ada-8f70-4914-9f77-eb5b98e6a43a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:49.300267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a70ba-8888-4a07-b33c-12f12b0528c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:49.300870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:49.300987 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a70ba-8888-4a07-b33c-12f12b0528c8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:49.301270 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a70ba-8888-4a07-b33c-12f12b0528c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:49.301539 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a70ba-8888-4a07-b33c-12f12b0528c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:49.324182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a70ba-8888-4a07-b33c-12f12b0528c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:49.325667 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a70ba-8888-4a07-b33c-12f12b0528c8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:49.347192 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b670853-6559-4276-98a7-d66f93c78d2f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-d34a70ba-8888-4a07-b33c-12f12b0528c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:49.367505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:49.368901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:49.370042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:49.370042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:49.370042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:49.370042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:49.371105 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 277/548] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:49.379043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a70ba-8888-4a07-b33c-12f12b0528c8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:49.421063 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f3c30-e9e9-4a21-9f7a-68d8c482ce8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:49.421705 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f3c30-e9e9-4a21-9f7a-68d8c482ce8d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:49.421965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f3c30-e9e9-4a21-9f7a-68d8c482ce8d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:49.422238 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f3c30-e9e9-4a21-9f7a-68d8c482ce8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:49.450748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f3c30-e9e9-4a21-9f7a-68d8c482ce8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-602f3c30-e9e9-4a21-9f7a-68d8c482ce8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:49.459042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:49.460033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:49.461384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:49.461384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:49.461384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:49.461384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:49.461384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:49.461384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:49.461384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:49.461384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:49.461384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:49.461384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:49.461384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:49.461384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:49.462187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 272/549] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:49.468874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f3c30-e9e9-4a21-9f7a-68d8c482ce8d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:49.531265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:49.565661 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a70ba-8888-4a07-b33c-12f12b0528c8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:49.627079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e257be8-ff8a-440a-b53e-a9fab36c9297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:49.627759 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e257be8-ff8a-440a-b53e-a9fab36c9297 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:49.627971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e257be8-ff8a-440a-b53e-a9fab36c9297 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:49.628244 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e257be8-ff8a-440a-b53e-a9fab36c9297 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:49.628942 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e257be8-ff8a-440a-b53e-a9fab36c9297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:49.629244 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e257be8-ff8a-440a-b53e-a9fab36c9297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:49.630621 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e257be8-ff8a-440a-b53e-a9fab36c9297 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:49.652677 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 278/550] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:49.663845 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e257be8-ff8a-440a-b53e-a9fab36c9297 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:49.665478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f3c30-e9e9-4a21-9f7a-68d8c482ce8d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:49.733117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1416955-cad3-41de-aed6-82d9f692f4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:49.733671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1416955-cad3-41de-aed6-82d9f692f4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:49.734010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1416955-cad3-41de-aed6-82d9f692f4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:49.734377 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1416955-cad3-41de-aed6-82d9f692f4f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:49.735043 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1416955-cad3-41de-aed6-82d9f692f4f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:49.735943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1416955-cad3-41de-aed6-82d9f692f4f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:49.736988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1416955-cad3-41de-aed6-82d9f692f4f9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:49.753910 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:49.763469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 273/551] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:00:49 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:49.772674 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1416955-cad3-41de-aed6-82d9f692f4f9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:49.844136 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e257be8-ff8a-440a-b53e-a9fab36c9297 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:49.879572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4aeb8-58ae-4f8b-a1a0-1a5830aa12cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:49.880224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4aeb8-58ae-4f8b-a1a0-1a5830aa12cc None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:49.880507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4aeb8-58ae-4f8b-a1a0-1a5830aa12cc None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:49.880786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4aeb8-58ae-4f8b-a1a0-1a5830aa12cc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:49.888228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4aeb8-58ae-4f8b-a1a0-1a5830aa12cc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:49.889674 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4aeb8-58ae-4f8b-a1a0-1a5830aa12cc None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-3fd4aeb8-58ae-4f8b-a1a0-1a5830aa12cc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:00:49.902211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:49.903291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:49.904874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:49.904874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:49.904874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:00:49.904874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:00:49.904874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:00:49.904874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:00:49.904874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:00:49.904874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:00:49.904874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:00:49.904874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:49.904874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 279/552] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:00:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:49.904874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee3dcf-7d10-4f24-bbad-00a0e28d0137 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:49.905498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee3dcf-7d10-4f24-bbad-00a0e28d0137 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:49.905498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee3dcf-7d10-4f24-bbad-00a0e28d0137 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:49.910650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 280/553] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:49.910650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee3dcf-7d10-4f24-bbad-00a0e28d0137 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:49.913556 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:49.935123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1416955-cad3-41de-aed6-82d9f692f4f9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:49.988245 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59a51c-3a5e-463a-ae70-d95f8619ec01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:49.989286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59a51c-3a5e-463a-ae70-d95f8619ec01 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:49.989815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59a51c-3a5e-463a-ae70-d95f8619ec01 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:49.990361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59a51c-3a5e-463a-ae70-d95f8619ec01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:50.009612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59a51c-3a5e-463a-ae70-d95f8619ec01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:50.011826 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b59a51c-3a5e-463a-ae70-d95f8619ec01 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1b59a51c-3a5e-463a-ae70-d95f8619ec01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:50.031378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:50.032309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:50.033279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:50.033279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:50.033279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:50.033279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:50.037157 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 274/554] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:50.038450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4eb33e-fb78-4047-a89e-075af7abebb7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:50.038760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4eb33e-fb78-4047-a89e-075af7abebb7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:50.039471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4eb33e-fb78-4047-a89e-075af7abebb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:50.043759 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee3dcf-7d10-4f24-bbad-00a0e28d0137 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:50.058794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0f4eb33e-fb78-4047-a89e-075af7abebb7 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:50.059985 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0f4eb33e-fb78-4047-a89e-075af7abebb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:50.088637 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27a077-685a-438b-882b-35aa6d5bb3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:50.089869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27a077-685a-438b-882b-35aa6d5bb3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:50.090497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27a077-685a-438b-882b-35aa6d5bb3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:50.091382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27a077-685a-438b-882b-35aa6d5bb3d0 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:50.092673 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27a077-685a-438b-882b-35aa6d5bb3d0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:50.093290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27a077-685a-438b-882b-35aa6d5bb3d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:50.095190 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27a077-685a-438b-882b-35aa6d5bb3d0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:50.114371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:50.119948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 281/555] 10.222.0.100 () {56 vars in 1140 bytes} [Mon Aug 1 15:00:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:50.121633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27a077-685a-438b-882b-35aa6d5bb3d0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:50.140162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 275/556] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:50.141395 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4eb33e-fb78-4047-a89e-075af7abebb7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:50.215033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:50.274363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27a077-685a-438b-882b-35aa6d5bb3d0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:50.278304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4eb33e-fb78-4047-a89e-075af7abebb7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:50.314009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49391fa-b4e2-4233-9382-20acba25e95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:50.315038 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49391fa-b4e2-4233-9382-20acba25e95c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:50.315500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49391fa-b4e2-4233-9382-20acba25e95c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:50.316023 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49391fa-b4e2-4233-9382-20acba25e95c None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:50.316439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f2122-04f6-4181-8fad-679f8683df25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:50.317119 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f2122-04f6-4181-8fad-679f8683df25 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:50.317407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49391fa-b4e2-4233-9382-20acba25e95c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:50.317510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f2122-04f6-4181-8fad-679f8683df25 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:50.317960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f2122-04f6-4181-8fad-679f8683df25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:50.318140 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49391fa-b4e2-4233-9382-20acba25e95c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:50.320373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49391fa-b4e2-4233-9382-20acba25e95c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:50.333630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 282/557] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:50.335395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49391fa-b4e2-4233-9382-20acba25e95c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:50.375838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:50.397885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f2122-04f6-4181-8fad-679f8683df25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:50.399481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f2122-04f6-4181-8fad-679f8683df25 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:50.449391 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49391fa-b4e2-4233-9382-20acba25e95c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:50.475664 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:50.488900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83116f26-e446-44f4-9595-f397391942db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:50.489870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83116f26-e446-44f4-9595-f397391942db None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:50.490360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83116f26-e446-44f4-9595-f397391942db None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:50.491501 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83116f26-e446-44f4-9595-f397391942db None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:50.503449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83116f26-e446-44f4-9595-f397391942db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-83116f26-e446-44f4-9595-f397391942db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:50.510046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:50.511525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:50.512382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:50.512382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:50.512382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:50.512382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:50.512382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:50.512382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:50.512382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:50.512382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:50.512382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:50.512382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:50.512382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:50.512382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:50.513013 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 276/558] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:50 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:50.513013 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f2122-04f6-4181-8fad-679f8683df25 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:50.515886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 283/559] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:50.517026 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83116f26-e446-44f4-9595-f397391942db None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:50.629125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83116f26-e446-44f4-9595-f397391942db None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:50.636829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:50.658752 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f2122-04f6-4181-8fad-679f8683df25 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:50.669357 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4fdec-fd87-4283-a8f7-a4c88eb763fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:50.670327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4fdec-fd87-4283-a8f7-a4c88eb763fb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:50.670955 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4fdec-fd87-4283-a8f7-a4c88eb763fb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:50.671448 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4fdec-fd87-4283-a8f7-a4c88eb763fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:50.684833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4fdec-fd87-4283-a8f7-a4c88eb763fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:50.686745 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4fdec-fd87-4283-a8f7-a4c88eb763fb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:50.699724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e0192-cbf1-45c5-8e2b-edbb2d429fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:50.700921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e0192-cbf1-45c5-8e2b-edbb2d429fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:50.701451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e0192-cbf1-45c5-8e2b-edbb2d429fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:50.701906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e0192-cbf1-45c5-8e2b-edbb2d429fe4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2bb4fdec-fd87-4283-a8f7-a4c88eb763fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:50.705427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:50.706421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:50.707342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:50.707342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:50.707342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:50.707342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:50.709735 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 284/560] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:50.710883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4fdec-fd87-4283-a8f7-a4c88eb763fb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:50.712300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e0192-cbf1-45c5-8e2b-edbb2d429fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:50.714197 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e0192-cbf1-45c5-8e2b-edbb2d429fe4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8d0e0192-cbf1-45c5-8e2b-edbb2d429fe4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:00:50.727547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:50.728567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:50.729541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:50.729541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:50.729541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:00:50.729541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:00:50.729541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:00:50.729541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:00:50.729541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:00:50.729541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:00:50.729541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:00:50.729541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:50.733391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 277/561] 10.222.0.100 () {56 vars in 1098 bytes} [Mon Aug 1 15:00:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:50.734560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e0192-cbf1-45c5-8e2b-edbb2d429fe4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:50.754903 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:50.830017 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4fdec-fd87-4283-a8f7-a4c88eb763fb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:50.850961 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e0192-cbf1-45c5-8e2b-edbb2d429fe4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:50.874960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17bdbd-074d-47a3-835f-42cbde5bd899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:50.875652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17bdbd-074d-47a3-835f-42cbde5bd899 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:50.876091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17bdbd-074d-47a3-835f-42cbde5bd899 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:50.876507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17bdbd-074d-47a3-835f-42cbde5bd899 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:50.877305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17bdbd-074d-47a3-835f-42cbde5bd899 None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:50.877621 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17bdbd-074d-47a3-835f-42cbde5bd899 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:50.879783 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17bdbd-074d-47a3-835f-42cbde5bd899 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:50.892835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb2bc1-0f22-4cb8-835f-3f4372874f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:50.893853 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb2bc1-0f22-4cb8-835f-3f4372874f50 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:50.894343 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb2bc1-0f22-4cb8-835f-3f4372874f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:50.894858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb2bc1-0f22-4cb8-835f-3f4372874f50 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:50.895775 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb2bc1-0f22-4cb8-835f-3f4372874f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:50.896302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb2bc1-0f22-4cb8-835f-3f4372874f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:50.898333 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 285/562] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:00:50 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:50.898818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb2bc1-0f22-4cb8-835f-3f4372874f50 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:50.899318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17bdbd-074d-47a3-835f-42cbde5bd899 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:50.914533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 278/563] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:50.916172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb2bc1-0f22-4cb8-835f-3f4372874f50 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:50.920054 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:51.010388 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:51.017882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17bdbd-074d-47a3-835f-42cbde5bd899 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:51.026138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb2bc1-0f22-4cb8-835f-3f4372874f50 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:51.055669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6335216-498d-4049-8427-0fecf6e99de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:51.056635 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6335216-498d-4049-8427-0fecf6e99de6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:51.057096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6335216-498d-4049-8427-0fecf6e99de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:51.057656 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6335216-498d-4049-8427-0fecf6e99de6 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:51.058772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6335216-498d-4049-8427-0fecf6e99de6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:51.059259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6335216-498d-4049-8427-0fecf6e99de6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:51.060910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6335216-498d-4049-8427-0fecf6e99de6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:51.062881 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99eca0-01a4-4b2d-9625-7848a961d1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:51.063528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99eca0-01a4-4b2d-9625-7848a961d1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:51.063817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99eca0-01a4-4b2d-9625-7848a961d1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:51.064098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99eca0-01a4-4b2d-9625-7848a961d1e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:51.075403 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99eca0-01a4-4b2d-9625-7848a961d1e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:51.076866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99eca0-01a4-4b2d-9625-7848a961d1e1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:51.081951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 286/564] 10.222.0.100 () {56 vars in 1138 bytes} [Mon Aug 1 15:00:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:51.083036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6335216-498d-4049-8427-0fecf6e99de6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2e99eca0-01a4-4b2d-9625-7848a961d1e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:51.096187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:51.097640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:51.098934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:51.098934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:51.098934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:51.098934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:51.098934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 279/565] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:51.098934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b236cd-84c9-41db-b8f2-5d4860e1caa9 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:51.099469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b236cd-84c9-41db-b8f2-5d4860e1caa9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:51.099469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b236cd-84c9-41db-b8f2-5d4860e1caa9 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:51.102703 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 280/566] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:51.102703 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b236cd-84c9-41db-b8f2-5d4860e1caa9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:51.161116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:51.200184 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6335216-498d-4049-8427-0fecf6e99de6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:51.209270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b236cd-84c9-41db-b8f2-5d4860e1caa9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:51.238173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c8993-5b20-4760-a858-b419664d50c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:51.238915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c8993-5b20-4760-a858-b419664d50c4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:51.239196 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c8993-5b20-4760-a858-b419664d50c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:51.239462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c8993-5b20-4760-a858-b419664d50c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:51.246090 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6ea9b-3a25-4887-8801-847552158619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:51.246676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c8993-5b20-4760-a858-b419664d50c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:51.246780 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6ea9b-3a25-4887-8801-847552158619 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:51.247028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6ea9b-3a25-4887-8801-847552158619 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:51.247304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6ea9b-3a25-4887-8801-847552158619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:51.248139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c8993-5b20-4760-a858-b419664d50c4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-432c8993-5b20-4760-a858-b419664d50c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:00:51.259962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:51.261004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:51.261951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:00:51.261951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:00:51.261951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:00:51.261951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:00:51.261951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:00:51.261951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:00:51.261951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:51.261951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:51.261951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 287/567] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:51.262436 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c8993-5b20-4760-a858-b419664d50c4 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:51.268256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6ea9b-3a25-4887-8801-847552158619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:51.269778 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6ea9b-3a25-4887-8801-847552158619 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:51.280551 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5bd6ea9b-3a25-4887-8801-847552158619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:51.289765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:51.291473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:51.293540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:51.293540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:51.293540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:51.293540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:51.293540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 281/568] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:51.293540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7ebab-3d03-4e28-9292-0d1ab231bf1b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:51.293540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7ebab-3d03-4e28-9292-0d1ab231bf1b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:51.293540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7ebab-3d03-4e28-9292-0d1ab231bf1b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:51.294429 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 282/569] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:51 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:51.295569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b1478-96ef-46ec-ab77-a8e0a2acbf28 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:51.295569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b1478-96ef-46ec-ab77-a8e0a2acbf28 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:51.295569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b1478-96ef-46ec-ab77-a8e0a2acbf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:51.313975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1b8b1478-96ef-46ec-ab77-a8e0a2acbf28 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:51.315201 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1b8b1478-96ef-46ec-ab77-a8e0a2acbf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:51.398153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 283/570] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:51.398153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b1478-96ef-46ec-ab77-a8e0a2acbf28 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:51.398477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c8993-5b20-4760-a858-b419664d50c4 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:51.429933 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147dd4b-e6f9-4466-a5aa-406279d0ae32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:51.430552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147dd4b-e6f9-4466-a5aa-406279d0ae32 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:51.430862 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147dd4b-e6f9-4466-a5aa-406279d0ae32 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:51.431134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147dd4b-e6f9-4466-a5aa-406279d0ae32 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:51.431817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147dd4b-e6f9-4466-a5aa-406279d0ae32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:51.432124 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147dd4b-e6f9-4466-a5aa-406279d0ae32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:51.433484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147dd4b-e6f9-4466-a5aa-406279d0ae32 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:51.438716 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:51.446308 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 288/571] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:51.448339 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af21f98-ffb9-47d1-a735-942edb82b37a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:51.448693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af21f98-ffb9-47d1-a735-942edb82b37a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:51.449180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af21f98-ffb9-47d1-a735-942edb82b37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:51.469418 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-8af21f98-ffb9-47d1-a735-942edb82b37a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:51.470630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8af21f98-ffb9-47d1-a735-942edb82b37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:51.523392 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b1478-96ef-46ec-ab77-a8e0a2acbf28 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:51.549382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 289/572] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:51.550249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af21f98-ffb9-47d1-a735-942edb82b37a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:51.554967 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:51.570538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b11674-168f-4bfb-bcaf-a64b326499c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:51.571301 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b11674-168f-4bfb-bcaf-a64b326499c5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:51.571517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b11674-168f-4bfb-bcaf-a64b326499c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:51.571776 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b11674-168f-4bfb-bcaf-a64b326499c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:51.572614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b11674-168f-4bfb-bcaf-a64b326499c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:51.573029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b11674-168f-4bfb-bcaf-a64b326499c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:51.574471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b11674-168f-4bfb-bcaf-a64b326499c5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:51.589384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 284/573] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:00:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:51.590062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b11674-168f-4bfb-bcaf-a64b326499c5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:51.664597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af21f98-ffb9-47d1-a735-942edb82b37a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:51.688878 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:51.696063 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b11674-168f-4bfb-bcaf-a64b326499c5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:51.699649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770fa4c9-18c5-4c9f-b351-1b828133a8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:51.700269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fa4c9-18c5-4c9f-b351-1b828133a8ae None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:51.700546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fa4c9-18c5-4c9f-b351-1b828133a8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:51.700820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770fa4c9-18c5-4c9f-b351-1b828133a8ae None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:51.701520 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770fa4c9-18c5-4c9f-b351-1b828133a8ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:51.701859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770fa4c9-18c5-4c9f-b351-1b828133a8ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:51.703336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770fa4c9-18c5-4c9f-b351-1b828133a8ae None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:51.715642 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 290/574] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:51.716717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770fa4c9-18c5-4c9f-b351-1b828133a8ae None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:51.736688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18acef-6e02-4f48-b2a3-d3a75f62519a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:51.737333 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18acef-6e02-4f48-b2a3-d3a75f62519a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:51.737608 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18acef-6e02-4f48-b2a3-d3a75f62519a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:51.737882 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18acef-6e02-4f48-b2a3-d3a75f62519a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:51.751273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18acef-6e02-4f48-b2a3-d3a75f62519a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7f18acef-6e02-4f48-b2a3-d3a75f62519a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:51.757222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:51.758367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:51.759438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:51.759438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:51.759438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:51.759438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:51.759438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:51.759438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:51.759438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:51.759438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:51.759438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:51.759438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:51.759438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:51.759438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:51.759438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 285/575] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:51.760149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18acef-6e02-4f48-b2a3-d3a75f62519a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:51.782921 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:51.825304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770fa4c9-18c5-4c9f-b351-1b828133a8ae None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:51.860091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5a62c-b5ed-414c-82e6-498d96907659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:51.860893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5a62c-b5ed-414c-82e6-498d96907659 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:51.861099 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5a62c-b5ed-414c-82e6-498d96907659 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:51.861387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5a62c-b5ed-414c-82e6-498d96907659 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:51.867584 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18acef-6e02-4f48-b2a3-d3a75f62519a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:51.869579 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5a62c-b5ed-414c-82e6-498d96907659 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:51.871072 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5a62c-b5ed-414c-82e6-498d96907659 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-4db5a62c-b5ed-414c-82e6-498d96907659 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:00:51.882387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:51.883407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:51.884373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:51.884373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:51.884373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:00:51.884373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:00:51.884373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:00:51.884373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:00:51.884373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:00:51.884373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:00:51.884373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:00:51.884373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:51.884373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 291/576] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:51.884373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5a62c-b5ed-414c-82e6-498d96907659 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:51.906971 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ef792-7221-4b16-9d4d-34c9cff7cc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:51.907658 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ef792-7221-4b16-9d4d-34c9cff7cc8c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:51.907908 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ef792-7221-4b16-9d4d-34c9cff7cc8c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:51.908186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ef792-7221-4b16-9d4d-34c9cff7cc8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:51.922729 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ef792-7221-4b16-9d4d-34c9cff7cc8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9a0ef792-7221-4b16-9d4d-34c9cff7cc8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:51.927980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:51.929453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:51.930304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:51.930304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:51.930304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:51.930304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:51.930304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:51.930304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:51.930304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:51.930304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:51.930304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:51.930304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:51.930304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:51.930304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:51.931485 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 286/577] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:51.932211 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ef792-7221-4b16-9d4d-34c9cff7cc8c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:51.956380 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:51.999957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5a62c-b5ed-414c-82e6-498d96907659 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:52.024444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:52.035574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb86226-204d-4cdc-85e4-c3c7c6ab92bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:52.036407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb86226-204d-4cdc-85e4-c3c7c6ab92bd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:52.036688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb86226-204d-4cdc-85e4-c3c7c6ab92bd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:52.036966 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb86226-204d-4cdc-85e4-c3c7c6ab92bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:52.038953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb86226-204d-4cdc-85e4-c3c7c6ab92bd None admin] RBAC: Adding query filter params (role.id=70e0c42f1bdd49178db398637f021bbc, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=41a399a3cc7446c6a8dad94075851797) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:52.039274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb86226-204d-4cdc-85e4-c3c7c6ab92bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:52.040736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb86226-204d-4cdc-85e4-c3c7c6ab92bd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:52.046143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 292/578] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:51 2022] GET /identity/v3/role_assignments?user.id=41a399a3cc7446c6a8dad94075851797&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=70e0c42f1bdd49178db398637f021bbc => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:52.046955 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb86226-204d-4cdc-85e4-c3c7c6ab92bd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:52.048760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ef792-7221-4b16-9d4d-34c9cff7cc8c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:52.086987 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9af0f7-20b5-49a0-b3bf-6df6b2936256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:52.087661 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9af0f7-20b5-49a0-b3bf-6df6b2936256 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:52.088211 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9af0f7-20b5-49a0-b3bf-6df6b2936256 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:52.088489 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9af0f7-20b5-49a0-b3bf-6df6b2936256 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:52.097338 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9af0f7-20b5-49a0-b3bf-6df6b2936256 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:52.098867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9af0f7-20b5-49a0-b3bf-6df6b2936256 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5a9af0f7-20b5-49a0-b3bf-6df6b2936256 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:00:52.112219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:52.113213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:52.114155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:00:52.114155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:00:52.114155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:00:52.114155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:00:52.114155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:00:52.114155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:00:52.114155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:52.114155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:52.114155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 287/579] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:52.114646 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9af0f7-20b5-49a0-b3bf-6df6b2936256 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:52.180067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb86226-204d-4cdc-85e4-c3c7c6ab92bd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:52.212152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:52.222109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7802b25-2db7-4d47-abb6-05f8cdcaf126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:52.222771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7802b25-2db7-4d47-abb6-05f8cdcaf126 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:52.223059 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7802b25-2db7-4d47-abb6-05f8cdcaf126 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:52.223336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7802b25-2db7-4d47-abb6-05f8cdcaf126 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:52.224120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7802b25-2db7-4d47-abb6-05f8cdcaf126 None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:52.224445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7802b25-2db7-4d47-abb6-05f8cdcaf126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:52.226464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7802b25-2db7-4d47-abb6-05f8cdcaf126 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:52.240056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 293/580] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:00:52 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:52.240056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7802b25-2db7-4d47-abb6-05f8cdcaf126 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:52.252932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9af0f7-20b5-49a0-b3bf-6df6b2936256 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:52.298107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f4ba4-207e-45d0-b97c-cb266818244a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:52.299164 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f4ba4-207e-45d0-b97c-cb266818244a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:52.299657 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f4ba4-207e-45d0-b97c-cb266818244a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:52.300094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f4ba4-207e-45d0-b97c-cb266818244a None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:52.301381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f4ba4-207e-45d0-b97c-cb266818244a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:52.301865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f4ba4-207e-45d0-b97c-cb266818244a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:52.304218 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f4ba4-207e-45d0-b97c-cb266818244a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:52.304650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:52.329936 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 288/581] 10.222.0.100 () {56 vars in 1142 bytes} [Mon Aug 1 15:00:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:52.331356 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f4ba4-207e-45d0-b97c-cb266818244a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:52.381795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7802b25-2db7-4d47-abb6-05f8cdcaf126 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:52.425069 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b6541-f29d-4861-b8d2-7333fb243932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:52.425702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b6541-f29d-4861-b8d2-7333fb243932 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:52.426041 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b6541-f29d-4861-b8d2-7333fb243932 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:52.426326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b6541-f29d-4861-b8d2-7333fb243932 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:52.427207 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b6541-f29d-4861-b8d2-7333fb243932 None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:52.427499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b6541-f29d-4861-b8d2-7333fb243932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:52.428899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b6541-f29d-4861-b8d2-7333fb243932 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:52.438048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 294/582] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:00:52 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:52.438909 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b6541-f29d-4861-b8d2-7333fb243932 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:52.471852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f4ba4-207e-45d0-b97c-cb266818244a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:52.514100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de13af-0995-4d55-9480-2ea042d13cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:52.514755 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de13af-0995-4d55-9480-2ea042d13cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:52.515045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de13af-0995-4d55-9480-2ea042d13cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:52.515324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8de13af-0995-4d55-9480-2ea042d13cf4 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:52.516101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de13af-0995-4d55-9480-2ea042d13cf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:52.516405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de13af-0995-4d55-9480-2ea042d13cf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:52.517880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8de13af-0995-4d55-9480-2ea042d13cf4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:52.519893 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:52.533520 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 289/583] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:52.534397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de13af-0995-4d55-9480-2ea042d13cf4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:52.565640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b6541-f29d-4861-b8d2-7333fb243932 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:52.581454 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:52.605733 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864cc3f-a2da-4d41-96b5-f970e5866a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:52.606386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864cc3f-a2da-4d41-96b5-f970e5866a81 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:52.606693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864cc3f-a2da-4d41-96b5-f970e5866a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:52.606974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864cc3f-a2da-4d41-96b5-f970e5866a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:52.619288 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864cc3f-a2da-4d41-96b5-f970e5866a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:52.620777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864cc3f-a2da-4d41-96b5-f970e5866a81 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-9864cc3f-a2da-4d41-96b5-f970e5866a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:52.636127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:52.637121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:52.638695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:52.638695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:52.638695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:52.638695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:52.638695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 295/584] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:52.638695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864cc3f-a2da-4d41-96b5-f970e5866a81 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:52.656007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8de13af-0995-4d55-9480-2ea042d13cf4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:52.692749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a3c14-0955-47c7-8698-9436cabceae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:52.693440 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a3c14-0955-47c7-8698-9436cabceae8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:52.693798 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a3c14-0955-47c7-8698-9436cabceae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:52.694150 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a3c14-0955-47c7-8698-9436cabceae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:52.705388 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a3c14-0955-47c7-8698-9436cabceae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:52.706962 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a3c14-0955-47c7-8698-9436cabceae8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-be4a3c14-0955-47c7-8698-9436cabceae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:52.722140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:52.723296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:52.724393 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:52.724393 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:52.724393 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:52.724393 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:52.730978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 290/585] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:52.730978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a3c14-0955-47c7-8698-9436cabceae8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:52.752538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864cc3f-a2da-4d41-96b5-f970e5866a81 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:52.785145 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:52.805226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481235e-c813-4010-b568-e914c262c91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:52.806230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481235e-c813-4010-b568-e914c262c91f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:52.806695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481235e-c813-4010-b568-e914c262c91f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:52.807179 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8481235e-c813-4010-b568-e914c262c91f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:52.825535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481235e-c813-4010-b568-e914c262c91f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:52.828172 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8481235e-c813-4010-b568-e914c262c91f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-8481235e-c813-4010-b568-e914c262c91f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:52.853880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:52.854816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:52.855713 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:52.855713 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:52.855713 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:52.855713 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:52.855713 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 296/586] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:52.856768 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481235e-c813-4010-b568-e914c262c91f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:52.869999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a3c14-0955-47c7-8698-9436cabceae8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:52.880241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:52.906472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45603c36-b42a-4c13-a0f0-46d0c80324da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:52.907206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45603c36-b42a-4c13-a0f0-46d0c80324da None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:52.907493 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45603c36-b42a-4c13-a0f0-46d0c80324da None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:52.907774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45603c36-b42a-4c13-a0f0-46d0c80324da None admin] PATH_INFO: `/v3/projects/d6e147b6607b481591ceb250f7208c16/users/ec50ac7c54324884a1b2a6c63d617907/roles/70e0c42f1bdd49178db398637f021bbc` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:52.923675 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45603c36-b42a-4c13-a0f0-46d0c80324da None admin] RBAC: Authorizing `identity:create_grant(project_id=d6e147b6607b481591ceb250f7208c16, user_id=ec50ac7c54324884a1b2a6c63d617907, role_id=70e0c42f1bdd49178db398637f021bbc)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:52.925124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45603c36-b42a-4c13-a0f0-46d0c80324da None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:52.938162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 291/587] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:00:52 2022] PUT /identity/v3/projects/d6e147b6607b481591ceb250f7208c16/users/ec50ac7c54324884a1b2a6c63d617907/roles/70e0c42f1bdd49178db398637f021bbc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:52.938987 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45603c36-b42a-4c13-a0f0-46d0c80324da None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:52.990486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8481235e-c813-4010-b568-e914c262c91f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:53.030507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f04aa4-8074-4a0c-84a8-9e16e5c8a253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:53.031196 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f04aa4-8074-4a0c-84a8-9e16e5c8a253 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:53.031549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f04aa4-8074-4a0c-84a8-9e16e5c8a253 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:53.031845 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f04aa4-8074-4a0c-84a8-9e16e5c8a253 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:53.032611 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f04aa4-8074-4a0c-84a8-9e16e5c8a253 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:53.032933 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f04aa4-8074-4a0c-84a8-9e16e5c8a253 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:53.034558 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f04aa4-8074-4a0c-84a8-9e16e5c8a253 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:53.053902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 297/588] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:53.055053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f04aa4-8074-4a0c-84a8-9e16e5c8a253 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:53.067427 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:53.078636 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45603c36-b42a-4c13-a0f0-46d0c80324da None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:53.129150 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8a41d-a722-4cff-a822-08825063450b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:53.129782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8a41d-a722-4cff-a822-08825063450b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:53.130060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8a41d-a722-4cff-a822-08825063450b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:53.130334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8a41d-a722-4cff-a822-08825063450b None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:53.131593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8a41d-a722-4cff-a822-08825063450b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:53.132024 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8a41d-a722-4cff-a822-08825063450b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:53.133463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8a41d-a722-4cff-a822-08825063450b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:53.148328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 292/589] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:53.149204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8a41d-a722-4cff-a822-08825063450b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:53.153372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:53.193175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f04aa4-8074-4a0c-84a8-9e16e5c8a253 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:53.234988 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a18259-0fb4-446d-81ce-0ed23a52ae02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:53.235946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18259-0fb4-446d-81ce-0ed23a52ae02 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:53.236209 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18259-0fb4-446d-81ce-0ed23a52ae02 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:53.236492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18259-0fb4-446d-81ce-0ed23a52ae02 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:53.237310 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a18259-0fb4-446d-81ce-0ed23a52ae02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:53.237651 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a18259-0fb4-446d-81ce-0ed23a52ae02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:53.239871 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a18259-0fb4-446d-81ce-0ed23a52ae02 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:53.252908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 298/590] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:53.258166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a18259-0fb4-446d-81ce-0ed23a52ae02 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:53.281847 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8a41d-a722-4cff-a822-08825063450b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:53.324586 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf536e7-d2d1-4ed8-86d2-14c5e9c64367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:53.325664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf536e7-d2d1-4ed8-86d2-14c5e9c64367 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:53.326113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf536e7-d2d1-4ed8-86d2-14c5e9c64367 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:53.326538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf536e7-d2d1-4ed8-86d2-14c5e9c64367 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:53.336871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf536e7-d2d1-4ed8-86d2-14c5e9c64367 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:53.338928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf536e7-d2d1-4ed8-86d2-14c5e9c64367 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5cf536e7-d2d1-4ed8-86d2-14c5e9c64367 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:00:53.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:53.360042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:53.362204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:53.362204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:53.362204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:00:53.362204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:00:53.362204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:00:53.362204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:00:53.362204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:00:53.362204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:00:53.362204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:00:53.362204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:53.366801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 293/591] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:53.374440 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf536e7-d2d1-4ed8-86d2-14c5e9c64367 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:53.390478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a18259-0fb4-446d-81ce-0ed23a52ae02 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:53.405887 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:53.449032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4fde5-2eb5-4c32-af39-227c5c0525e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:53.450105 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4fde5-2eb5-4c32-af39-227c5c0525e3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:53.450564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4fde5-2eb5-4c32-af39-227c5c0525e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:53.453398 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4fde5-2eb5-4c32-af39-227c5c0525e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:53.458260 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:53.469208 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4fde5-2eb5-4c32-af39-227c5c0525e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:53.471113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4fde5-2eb5-4c32-af39-227c5c0525e3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-d4a4fde5-2eb5-4c32-af39-227c5c0525e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:00:53.484963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:53.485993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:53.486935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:00:53.486935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:00:53.486935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:00:53.486935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:00:53.486935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:00:53.486935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:00:53.486935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:53.486935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:53.489403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 299/592] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:53.490519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4fde5-2eb5-4c32-af39-227c5c0525e3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:53.530296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf536e7-d2d1-4ed8-86d2-14c5e9c64367 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:53.573963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3f7cb-bad6-453c-a422-e72adfe56ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:53.575497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f7cb-bad6-453c-a422-e72adfe56ddd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:53.575976 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f7cb-bad6-453c-a422-e72adfe56ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:53.576440 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f7cb-bad6-453c-a422-e72adfe56ddd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:53.586843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3f7cb-bad6-453c-a422-e72adfe56ddd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:53.589025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3f7cb-bad6-453c-a422-e72adfe56ddd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-bdb3f7cb-bad6-453c-a422-e72adfe56ddd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:00:53.606296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:53.607457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:53.608609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:53.608609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:53.608609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:00:53.608609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:00:53.608609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:00:53.608609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:00:53.608609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:00:53.608609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:00:53.608609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:00:53.608609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:53.615554 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 294/593] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:53.616831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3f7cb-bad6-453c-a422-e72adfe56ddd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:53.627224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4fde5-2eb5-4c32-af39-227c5c0525e3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:53.674588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b752c-6fb5-4be3-a52c-9eb2bdb591bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:53.675909 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b752c-6fb5-4be3-a52c-9eb2bdb591bf None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:53.676316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b752c-6fb5-4be3-a52c-9eb2bdb591bf None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:53.676666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b752c-6fb5-4be3-a52c-9eb2bdb591bf None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:53.678039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b752c-6fb5-4be3-a52c-9eb2bdb591bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:53.678429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b752c-6fb5-4be3-a52c-9eb2bdb591bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:53.680326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b752c-6fb5-4be3-a52c-9eb2bdb591bf None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:53.706224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 300/594] 10.222.0.100 () {56 vars in 1142 bytes} [Mon Aug 1 15:00:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:53.707542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b752c-6fb5-4be3-a52c-9eb2bdb591bf None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:53.718339 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:53.745701 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3f7cb-bad6-453c-a422-e72adfe56ddd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:53.783682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b31178-e636-402c-a7e4-149b18ba73f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:53.784642 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b31178-e636-402c-a7e4-149b18ba73f8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:53.785154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b31178-e636-402c-a7e4-149b18ba73f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:53.785641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b31178-e636-402c-a7e4-149b18ba73f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:53.786606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b31178-e636-402c-a7e4-149b18ba73f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:53.787229 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b31178-e636-402c-a7e4-149b18ba73f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:53.788444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:53.790261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b31178-e636-402c-a7e4-149b18ba73f8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:53.810075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 295/595] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:00:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:53.810075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b31178-e636-402c-a7e4-149b18ba73f8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:53.845890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b752c-6fb5-4be3-a52c-9eb2bdb591bf None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:53.884478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729e22f-fde3-4f22-b29e-44a4aa1aeb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:53.885105 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729e22f-fde3-4f22-b29e-44a4aa1aeb68 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:53.885393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729e22f-fde3-4f22-b29e-44a4aa1aeb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:53.885674 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729e22f-fde3-4f22-b29e-44a4aa1aeb68 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:53.886477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729e22f-fde3-4f22-b29e-44a4aa1aeb68 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:53.886836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729e22f-fde3-4f22-b29e-44a4aa1aeb68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:53.888298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729e22f-fde3-4f22-b29e-44a4aa1aeb68 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:53.907403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 301/596] 10.222.0.100 () {56 vars in 1144 bytes} [Mon Aug 1 15:00:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:53.908247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729e22f-fde3-4f22-b29e-44a4aa1aeb68 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:53.931390 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b31178-e636-402c-a7e4-149b18ba73f8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:53.971337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ada3a3-be1b-4198-8214-0ed2f904ab0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:53.972255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ada3a3-be1b-4198-8214-0ed2f904ab0a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:53.972633 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ada3a3-be1b-4198-8214-0ed2f904ab0a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:53.973023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ada3a3-be1b-4198-8214-0ed2f904ab0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:53.976252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:53.987065 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ada3a3-be1b-4198-8214-0ed2f904ab0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:53.989025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ada3a3-be1b-4198-8214-0ed2f904ab0a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e3ada3a3-be1b-4198-8214-0ed2f904ab0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:54.005694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:54.007198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:54.008115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:54.008115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:54.008115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:54.008115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:54.010413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 296/597] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:54.011511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ada3a3-be1b-4198-8214-0ed2f904ab0a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:54.022493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729e22f-fde3-4f22-b29e-44a4aa1aeb68 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:54.037146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:54.074840 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877466a-5c2f-4b8e-b6f6-6ec75b2da679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:54.075970 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877466a-5c2f-4b8e-b6f6-6ec75b2da679 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:54.076441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877466a-5c2f-4b8e-b6f6-6ec75b2da679 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:54.076897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877466a-5c2f-4b8e-b6f6-6ec75b2da679 None admin] PATH_INFO: `/v3/projects/d6e147b6607b481591ceb250f7208c16/users/ead1b36d35f3485f89edd15b8443feea/roles/70e0c42f1bdd49178db398637f021bbc` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:54.101789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877466a-5c2f-4b8e-b6f6-6ec75b2da679 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6e147b6607b481591ceb250f7208c16, user_id=ead1b36d35f3485f89edd15b8443feea, role_id=70e0c42f1bdd49178db398637f021bbc)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:54.102971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877466a-5c2f-4b8e-b6f6-6ec75b2da679 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:54.119607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 302/598] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:00:53 2022] PUT /identity/v3/projects/d6e147b6607b481591ceb250f7208c16/users/ead1b36d35f3485f89edd15b8443feea/roles/70e0c42f1bdd49178db398637f021bbc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:54.128274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877466a-5c2f-4b8e-b6f6-6ec75b2da679 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:54.137371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ada3a3-be1b-4198-8214-0ed2f904ab0a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:54.183391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a79be-eafc-4504-814d-674b7df79601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:54.184539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a79be-eafc-4504-814d-674b7df79601 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:54.185139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a79be-eafc-4504-814d-674b7df79601 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:54.185640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a79be-eafc-4504-814d-674b7df79601 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:54.200865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a79be-eafc-4504-814d-674b7df79601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:54.202894 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a79be-eafc-4504-814d-674b7df79601 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-af2a79be-eafc-4504-814d-674b7df79601 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:54.221010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:54.222003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:54.223399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:54.223399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:54.223399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:54.223399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:54.225678 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 297/599] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:54.232056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682d16c-4342-4c19-a8fa-cf94e9263fd4 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:54.232623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682d16c-4342-4c19-a8fa-cf94e9263fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:54.233128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682d16c-4342-4c19-a8fa-cf94e9263fd4 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:54.235061 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 298/600] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:54 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:54.236108 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682d16c-4342-4c19-a8fa-cf94e9263fd4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:54.278399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877466a-5c2f-4b8e-b6f6-6ec75b2da679 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:54.293286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:54.324699 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1232c-c06b-4548-803e-a3d483125fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:54.325718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1232c-c06b-4548-803e-a3d483125fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:54.326317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1232c-c06b-4548-803e-a3d483125fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:54.327118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1232c-c06b-4548-803e-a3d483125fc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:54.328104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1232c-c06b-4548-803e-a3d483125fc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:54.328615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1232c-c06b-4548-803e-a3d483125fc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:54.330271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1232c-c06b-4548-803e-a3d483125fc1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:54.344993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 303/601] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:54.346684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb095e-9191-4994-9133-95152f544a0a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:54.347228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb095e-9191-4994-9133-95152f544a0a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:54.347702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb095e-9191-4994-9133-95152f544a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:54.361817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682d16c-4342-4c19-a8fa-cf94e9263fd4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:54.368211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e3cb095e-9191-4994-9133-95152f544a0a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:54.369756 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e3cb095e-9191-4994-9133-95152f544a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:54.389458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:54.402805 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec982931-f7c3-4de4-913f-93ed4fa025ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:54.403346 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec982931-f7c3-4de4-913f-93ed4fa025ae None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:54.404235 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec982931-f7c3-4de4-913f-93ed4fa025ae None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:54.404235 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec982931-f7c3-4de4-913f-93ed4fa025ae None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:54.404721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec982931-f7c3-4de4-913f-93ed4fa025ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:54.405032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec982931-f7c3-4de4-913f-93ed4fa025ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:54.406416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec982931-f7c3-4de4-913f-93ed4fa025ae None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:54.420764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 299/602] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:54.421633 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec982931-f7c3-4de4-913f-93ed4fa025ae None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:54.465172 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 304/603] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:54.466353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb095e-9191-4994-9133-95152f544a0a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:54.545844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:54.555170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec982931-f7c3-4de4-913f-93ed4fa025ae None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:54.596285 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8082a-f50c-43e6-8168-2be5f499ad12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:54.596927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8082a-f50c-43e6-8168-2be5f499ad12 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:54.597193 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8082a-f50c-43e6-8168-2be5f499ad12 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:54.597476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8082a-f50c-43e6-8168-2be5f499ad12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:54.605763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8082a-f50c-43e6-8168-2be5f499ad12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:54.607328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8082a-f50c-43e6-8168-2be5f499ad12 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:54.619478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb095e-9191-4994-9133-95152f544a0a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-24c8082a-f50c-43e6-8168-2be5f499ad12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:00:54.620152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:54.621029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:54.621921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:00:54.621921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:00:54.621921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:00:54.621921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:00:54.621921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:00:54.621921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:00:54.621921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:54.621921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:54.621921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 300/604] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:54.622874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8082a-f50c-43e6-8168-2be5f499ad12 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:54.673139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5de3f-a8de-4fd4-95e3-7ebbe9d411d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:54.674624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5de3f-a8de-4fd4-95e3-7ebbe9d411d6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:54.675564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5de3f-a8de-4fd4-95e3-7ebbe9d411d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:54.676225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5de3f-a8de-4fd4-95e3-7ebbe9d411d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:54.686769 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5de3f-a8de-4fd4-95e3-7ebbe9d411d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:54.688761 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5de3f-a8de-4fd4-95e3-7ebbe9d411d6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-9da5de3f-a8de-4fd4-95e3-7ebbe9d411d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:00:54.706091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:54.707045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:54.707932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:00:54.707932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:00:54.707932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:00:54.707932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:00:54.707932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:00:54.707932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:00:54.707932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:54.707932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:54.715837 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 305/605] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:54.715837 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5de3f-a8de-4fd4-95e3-7ebbe9d411d6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:54.737259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:54.743058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8082a-f50c-43e6-8168-2be5f499ad12 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:54.779283 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5b1c8-8aaf-4036-b0aa-900d10fc4ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:54.779732 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5b1c8-8aaf-4036-b0aa-900d10fc4ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:54.780172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5b1c8-8aaf-4036-b0aa-900d10fc4ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:54.780437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c5b1c8-8aaf-4036-b0aa-900d10fc4ac8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:54.793501 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c5b1c8-8aaf-4036-b0aa-900d10fc4ac8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-33c5b1c8-8aaf-4036-b0aa-900d10fc4ac8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:54.798510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:54.799470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:54.800314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:54.800314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:54.800314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:54.800314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:54.800314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:54.800314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:54.800314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:54.800314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:54.800314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:54.800314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:54.800314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:54.800314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:54.800314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 301/606] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:54.801003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5b1c8-8aaf-4036-b0aa-900d10fc4ac8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:54.825750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:54.862389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5de3f-a8de-4fd4-95e3-7ebbe9d411d6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:54.899034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050f952-09e1-4c2c-afcb-5077d68fce3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:54.899708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050f952-09e1-4c2c-afcb-5077d68fce3b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:54.899989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050f952-09e1-4c2c-afcb-5077d68fce3b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:54.900257 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050f952-09e1-4c2c-afcb-5077d68fce3b None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:54.900995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050f952-09e1-4c2c-afcb-5077d68fce3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:54.901300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050f952-09e1-4c2c-afcb-5077d68fce3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:54.902883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050f952-09e1-4c2c-afcb-5077d68fce3b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:54.915756 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 306/607] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:00:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:54.916893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050f952-09e1-4c2c-afcb-5077d68fce3b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:54.925728 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c5b1c8-8aaf-4036-b0aa-900d10fc4ac8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:54.969662 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18835093-171f-4cea-ac66-13e82e6f29bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:54.970311 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18835093-171f-4cea-ac66-13e82e6f29bc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:54.970632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18835093-171f-4cea-ac66-13e82e6f29bc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:54.970930 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18835093-171f-4cea-ac66-13e82e6f29bc None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:54.971738 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18835093-171f-4cea-ac66-13e82e6f29bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:54.972095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18835093-171f-4cea-ac66-13e82e6f29bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:54.973611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18835093-171f-4cea-ac66-13e82e6f29bc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:54.992440 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 302/608] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:00:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:54.992440 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18835093-171f-4cea-ac66-13e82e6f29bc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:55.021331 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:55.064222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050f952-09e1-4c2c-afcb-5077d68fce3b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:55.116228 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:55.122799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684ea01-e988-466c-b616-dbd7d142b979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:55.123919 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684ea01-e988-466c-b616-dbd7d142b979 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:55.124361 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684ea01-e988-466c-b616-dbd7d142b979 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:55.124676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e684ea01-e988-466c-b616-dbd7d142b979 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:55.125545 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684ea01-e988-466c-b616-dbd7d142b979 None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:55.125882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684ea01-e988-466c-b616-dbd7d142b979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:55.127294 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e684ea01-e988-466c-b616-dbd7d142b979 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:55.136920 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 307/609] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:00:54 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:55.137857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684ea01-e988-466c-b616-dbd7d142b979 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:55.144117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18835093-171f-4cea-ac66-13e82e6f29bc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:55.192565 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f9ef8-648f-44c7-8737-e7c667ffffb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:55.193714 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f9ef8-648f-44c7-8737-e7c667ffffb6 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:55.194379 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f9ef8-648f-44c7-8737-e7c667ffffb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:55.194924 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f9ef8-648f-44c7-8737-e7c667ffffb6 None admin] PATH_INFO: `/v3/projects/d6e147b6607b481591ceb250f7208c16/users/e75edfc5bb404d63b27a355d5ab8af01/roles/70e0c42f1bdd49178db398637f021bbc` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:55.214991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f9ef8-648f-44c7-8737-e7c667ffffb6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6e147b6607b481591ceb250f7208c16, user_id=e75edfc5bb404d63b27a355d5ab8af01, role_id=70e0c42f1bdd49178db398637f021bbc)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:55.216462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f9ef8-648f-44c7-8737-e7c667ffffb6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:55.233862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 303/610] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:00:54 2022] PUT /identity/v3/projects/d6e147b6607b481591ceb250f7208c16/users/e75edfc5bb404d63b27a355d5ab8af01/roles/70e0c42f1bdd49178db398637f021bbc => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:55.235141 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f9ef8-648f-44c7-8737-e7c667ffffb6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:55.275651 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e684ea01-e988-466c-b616-dbd7d142b979 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:55.325055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce9867-df41-465b-83e6-b9f92d7b9fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:55.326388 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce9867-df41-465b-83e6-b9f92d7b9fc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:55.327088 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce9867-df41-465b-83e6-b9f92d7b9fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:55.327547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce9867-df41-465b-83e6-b9f92d7b9fc9 None admin] PATH_INFO: `/v3/projects/d6e147b6607b481591ceb250f7208c16/users/e7f757eabcb84de6ab042e0ac5a8158c/roles/70e0c42f1bdd49178db398637f021bbc` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:55.330983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:55.355909 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce9867-df41-465b-83e6-b9f92d7b9fc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6e147b6607b481591ceb250f7208c16, user_id=e7f757eabcb84de6ab042e0ac5a8158c, role_id=70e0c42f1bdd49178db398637f021bbc)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:55.357464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce9867-df41-465b-83e6-b9f92d7b9fc9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:55.386710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 308/611] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:00:55 2022] PUT /identity/v3/projects/d6e147b6607b481591ceb250f7208c16/users/e7f757eabcb84de6ab042e0ac5a8158c/roles/70e0c42f1bdd49178db398637f021bbc => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:55.386710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51db03-b6bc-40ee-99cc-9bdb5b0ac85f None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:55.386710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51db03-b6bc-40ee-99cc-9bdb5b0ac85f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:55.386710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51db03-b6bc-40ee-99cc-9bdb5b0ac85f None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:55.392742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 309/612] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:55 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:55.394609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081eaa3-f5ca-43dd-9901-81365129c72e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:55.394937 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081eaa3-f5ca-43dd-9901-81365129c72e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:55.395220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081eaa3-f5ca-43dd-9901-81365129c72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:55.417229 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f9ef8-648f-44c7-8737-e7c667ffffb6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:55.424240 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e081eaa3-f5ca-43dd-9901-81365129c72e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:55.428812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e081eaa3-f5ca-43dd-9901-81365129c72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:55.471085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:55.481719 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0beb1-a9dd-479d-93b6-b5e9f7ce3ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:55.482593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0beb1-a9dd-479d-93b6-b5e9f7ce3ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:55.482995 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0beb1-a9dd-479d-93b6-b5e9f7ce3ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:55.483285 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0beb1-a9dd-479d-93b6-b5e9f7ce3ed0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:55.498455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0beb1-a9dd-479d-93b6-b5e9f7ce3ed0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:55.500454 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e0beb1-a9dd-479d-93b6-b5e9f7ce3ed0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b9e0beb1-a9dd-479d-93b6-b5e9f7ce3ed0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:55.520222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:55.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:55.522237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:55.522237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:55.522237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:55.522237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:55.522237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 304/613] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:55.528394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0beb1-a9dd-479d-93b6-b5e9f7ce3ed0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:55.529839 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 310/614] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:55 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:00:55.557951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081eaa3-f5ca-43dd-9901-81365129c72e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:55.654059 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e0beb1-a9dd-479d-93b6-b5e9f7ce3ed0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:55.674457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081eaa3-f5ca-43dd-9901-81365129c72e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:55.691480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:55.694597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867ee79-95ee-42a0-ba6c-45fc3557fa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:55.695400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867ee79-95ee-42a0-ba6c-45fc3557fa64 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:55.695623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867ee79-95ee-42a0-ba6c-45fc3557fa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:55.696209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867ee79-95ee-42a0-ba6c-45fc3557fa64 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:55.697350 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867ee79-95ee-42a0-ba6c-45fc3557fa64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:55.697597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867ee79-95ee-42a0-ba6c-45fc3557fa64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:55.699017 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867ee79-95ee-42a0-ba6c-45fc3557fa64 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:55.715496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 305/615] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:00:55.720716 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867ee79-95ee-42a0-ba6c-45fc3557fa64 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:55.721638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c43a8-e302-48b6-ab93-975e7b046e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:55.721638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c43a8-e302-48b6-ab93-975e7b046e1c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:55.721638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c43a8-e302-48b6-ab93-975e7b046e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:55.722138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c43a8-e302-48b6-ab93-975e7b046e1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:55.747592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924c43a8-e302-48b6-ab93-975e7b046e1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-924c43a8-e302-48b6-ab93-975e7b046e1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:55.760886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:55.763147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:55.765296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:55.765296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:55.765296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:55.765296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:55.765296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:55.765296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:55.765296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:55.765296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:55.765296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:55.765296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:55.765296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:55.765296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:55.765296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 311/616] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:55.768571 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c43a8-e302-48b6-ab93-975e7b046e1c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:55.768762 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:55.832220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867ee79-95ee-42a0-ba6c-45fc3557fa64 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:55.878543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7036a3-8aa7-43d9-88b2-013e7558721c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:55.879232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7036a3-8aa7-43d9-88b2-013e7558721c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:55.879512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7036a3-8aa7-43d9-88b2-013e7558721c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:55.879779 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7036a3-8aa7-43d9-88b2-013e7558721c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:55.891788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7036a3-8aa7-43d9-88b2-013e7558721c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:55.893254 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7036a3-8aa7-43d9-88b2-013e7558721c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-ef7036a3-8aa7-43d9-88b2-013e7558721c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:00:55.915231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:55.918683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:55.922156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:55.922156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:55.922156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:00:55.922156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:00:55.922156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:00:55.922156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:00:55.922156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:00:55.922156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:00:55.922156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:00:55.922156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:55.930612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 306/617] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:00:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:55.931591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7036a3-8aa7-43d9-88b2-013e7558721c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:55.975804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c43a8-e302-48b6-ab93-975e7b046e1c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:56.081569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7036a3-8aa7-43d9-88b2-013e7558721c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:56.083673 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:56.091092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c3fc5-9d28-4791-a0b2-5560b80a4fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:56.091776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c3fc5-9d28-4791-a0b2-5560b80a4fac None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:56.092057 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c3fc5-9d28-4791-a0b2-5560b80a4fac None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:56.092363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c3fc5-9d28-4791-a0b2-5560b80a4fac None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:56.093914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c3fc5-9d28-4791-a0b2-5560b80a4fac None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:56.094318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c3fc5-9d28-4791-a0b2-5560b80a4fac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:56.097286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c3fc5-9d28-4791-a0b2-5560b80a4fac None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:56.120174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 312/618] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:00:55 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:56.120769 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c3fc5-9d28-4791-a0b2-5560b80a4fac None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:56.162010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:56.172434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b88f25-c81e-4ce8-bb90-b796fbd91ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:56.172991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b88f25-c81e-4ce8-bb90-b796fbd91ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:56.173276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b88f25-c81e-4ce8-bb90-b796fbd91ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:56.173552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b88f25-c81e-4ce8-bb90-b796fbd91ab6 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:56.174363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b88f25-c81e-4ce8-bb90-b796fbd91ab6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:56.174692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b88f25-c81e-4ce8-bb90-b796fbd91ab6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:56.176131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b88f25-c81e-4ce8-bb90-b796fbd91ab6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:56.210943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 307/619] 10.222.0.100 () {56 vars in 1140 bytes} [Mon Aug 1 15:00:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:56.216513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b88f25-c81e-4ce8-bb90-b796fbd91ab6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:56.237000 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c3fc5-9d28-4791-a0b2-5560b80a4fac None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:56.274515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c4a91-2be2-4fc1-9bd3-a4f9e8ebc0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:56.275202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c4a91-2be2-4fc1-9bd3-a4f9e8ebc0bd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:56.275488 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c4a91-2be2-4fc1-9bd3-a4f9e8ebc0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:56.275763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c4a91-2be2-4fc1-9bd3-a4f9e8ebc0bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:56.288181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c4a91-2be2-4fc1-9bd3-a4f9e8ebc0bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:56.289841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c4a91-2be2-4fc1-9bd3-a4f9e8ebc0bd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-c66c4a91-2be2-4fc1-9bd3-a4f9e8ebc0bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:56.306087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:56.307092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:56.307977 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:56.307977 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:56.307977 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:56.307977 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:56.311976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 313/620] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:56.313236 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c4a91-2be2-4fc1-9bd3-a4f9e8ebc0bd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:56.414153 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:56.429773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c4a91-2be2-4fc1-9bd3-a4f9e8ebc0bd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:56.473978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b88f25-c81e-4ce8-bb90-b796fbd91ab6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:56.493311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74627fe8-d5bd-418c-93fd-2aed6b598ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:56.494194 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74627fe8-d5bd-418c-93fd-2aed6b598ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:56.494486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74627fe8-d5bd-418c-93fd-2aed6b598ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:56.494815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74627fe8-d5bd-418c-93fd-2aed6b598ab4 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:56.496017 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74627fe8-d5bd-418c-93fd-2aed6b598ab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:56.496401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74627fe8-d5bd-418c-93fd-2aed6b598ab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:56.498101 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74627fe8-d5bd-418c-93fd-2aed6b598ab4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:56.519206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 314/621] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:56.519206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c8cdd-c939-4a0a-a59c-4ad086c5eb28 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:56.519206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c8cdd-c939-4a0a-a59c-4ad086c5eb28 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:56.519206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c8cdd-c939-4a0a-a59c-4ad086c5eb28 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:56.519206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 315/622] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:56.522835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c8cdd-c939-4a0a-a59c-4ad086c5eb28 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:56.529689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c72b5d-6ec2-4505-b4ec-775f06df2610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:56.530363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c72b5d-6ec2-4505-b4ec-775f06df2610 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:56.530645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c72b5d-6ec2-4505-b4ec-775f06df2610 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:56.530966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c72b5d-6ec2-4505-b4ec-775f06df2610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:56.552288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c72b5d-6ec2-4505-b4ec-775f06df2610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:56.554308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c72b5d-6ec2-4505-b4ec-775f06df2610 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-26c72b5d-6ec2-4505-b4ec-775f06df2610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:56.571359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:56.572353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:56.573915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:56.573915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:56.573915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:56.573915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:56.573915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 308/623] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:56.574486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2b589-04a6-44fa-9837-d9df6369160c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:56.574823 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2b589-04a6-44fa-9837-d9df6369160c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:56.575105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2b589-04a6-44fa-9837-d9df6369160c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:56.596484 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-acf2b589-04a6-44fa-9837-d9df6369160c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:56.597855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-acf2b589-04a6-44fa-9837-d9df6369160c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:56.607052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:56.643530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c8cdd-c939-4a0a-a59c-4ad086c5eb28 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:56.678774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 309/624] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:56.680428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2b589-04a6-44fa-9837-d9df6369160c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:56.689526 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a5d9c-9642-4303-a1f9-087839920fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:56.690499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a5d9c-9642-4303-a1f9-087839920fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:56.691065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a5d9c-9642-4303-a1f9-087839920fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:56.691574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a5d9c-9642-4303-a1f9-087839920fd6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:56.707619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a5d9c-9642-4303-a1f9-087839920fd6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:56.709641 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a5d9c-9642-4303-a1f9-087839920fd6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-603a5d9c-9642-4303-a1f9-087839920fd6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:00:56.723006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:56.724061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:56.725622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:56.725622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:56.725622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:00:56.725622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:00:56.725622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:00:56.725622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:00:56.725622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:00:56.725622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:00:56.725622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:00:56.725622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:56.730330 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 316/625] 10.222.0.100 () {56 vars in 1098 bytes} [Mon Aug 1 15:00:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:00:56.731272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a5d9c-9642-4303-a1f9-087839920fd6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:56.744340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:56.816050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2b589-04a6-44fa-9837-d9df6369160c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:56.848326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a5d9c-9642-4303-a1f9-087839920fd6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:56.857474 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13314803-8dd2-42ef-83c2-22c21241b060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:56.858792 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13314803-8dd2-42ef-83c2-22c21241b060 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:56.859605 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13314803-8dd2-42ef-83c2-22c21241b060 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:56.860112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13314803-8dd2-42ef-83c2-22c21241b060 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:56.861226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13314803-8dd2-42ef-83c2-22c21241b060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:56.861688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13314803-8dd2-42ef-83c2-22c21241b060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:56.863838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13314803-8dd2-42ef-83c2-22c21241b060 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:56.879740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 310/626] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:56.881349 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13314803-8dd2-42ef-83c2-22c21241b060 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:56.889365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3581450-0049-4617-82d6-4c2072e52afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:56.890105 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3581450-0049-4617-82d6-4c2072e52afb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:56.890452 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3581450-0049-4617-82d6-4c2072e52afb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:56.890824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3581450-0049-4617-82d6-4c2072e52afb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:56.894026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:56.906045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3581450-0049-4617-82d6-4c2072e52afb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f3581450-0049-4617-82d6-4c2072e52afb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:56.913560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:56.914750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:56.915815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:56.915815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:56.915815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:56.915815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:56.915815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:56.915815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:56.915815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:56.915815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:56.915815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:56.915815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:56.915815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:56.915815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:56.921830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 317/627] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:56.921830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3581450-0049-4617-82d6-4c2072e52afb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:57.010132 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13314803-8dd2-42ef-83c2-22c21241b060 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:57.013891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:57.049443 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a604d-f349-4121-aa4e-2becd9c97ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:57.050097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a604d-f349-4121-aa4e-2becd9c97ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:57.050377 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a604d-f349-4121-aa4e-2becd9c97ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:57.050648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a604d-f349-4121-aa4e-2becd9c97ed6 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:57.051486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a604d-f349-4121-aa4e-2becd9c97ed6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:57.051793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a604d-f349-4121-aa4e-2becd9c97ed6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:57.053307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a604d-f349-4121-aa4e-2becd9c97ed6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:57.059827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3581450-0049-4617-82d6-4c2072e52afb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:57.075215 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 311/628] 10.222.0.100 () {56 vars in 1138 bytes} [Mon Aug 1 15:00:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:57.075933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a604d-f349-4121-aa4e-2becd9c97ed6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:57.107093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9f87c-08d0-43ce-9e95-d54dd8bfd545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:57.108286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9f87c-08d0-43ce-9e95-d54dd8bfd545 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:57.108777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9f87c-08d0-43ce-9e95-d54dd8bfd545 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:57.109550 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9f87c-08d0-43ce-9e95-d54dd8bfd545 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:57.119204 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9f87c-08d0-43ce-9e95-d54dd8bfd545 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:57.121018 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9f87c-08d0-43ce-9e95-d54dd8bfd545 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-89a9f87c-08d0-43ce-9e95-d54dd8bfd545 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:00:57.137141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:57.138135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:57.139082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:00:57.139082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:00:57.139082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:00:57.139082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:00:57.139082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:00:57.139082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:00:57.139082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:57.139082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:57.142298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 318/629] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:57.143399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9f87c-08d0-43ce-9e95-d54dd8bfd545 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:57.153175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:57.207894 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a604d-f349-4121-aa4e-2becd9c97ed6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:57.246677 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b1e0d-cd68-42b9-82e3-eaa1ec2a15a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:57.247329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b1e0d-cd68-42b9-82e3-eaa1ec2a15a1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:57.247762 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b1e0d-cd68-42b9-82e3-eaa1ec2a15a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:57.248043 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b1e0d-cd68-42b9-82e3-eaa1ec2a15a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:57.248957 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b1e0d-cd68-42b9-82e3-eaa1ec2a15a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:57.249263 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b1e0d-cd68-42b9-82e3-eaa1ec2a15a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:57.250897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b1e0d-cd68-42b9-82e3-eaa1ec2a15a1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:57.265261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 312/630] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:00:57 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:57.265261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b1e0d-cd68-42b9-82e3-eaa1ec2a15a1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:57.284323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9f87c-08d0-43ce-9e95-d54dd8bfd545 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:57.319271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:57.329963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f012af-60e0-40b0-8e54-93fcb9839446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:57.331141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f012af-60e0-40b0-8e54-93fcb9839446 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:57.331627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f012af-60e0-40b0-8e54-93fcb9839446 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:57.332499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f012af-60e0-40b0-8e54-93fcb9839446 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:57.395425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b1e0d-cd68-42b9-82e3-eaa1ec2a15a1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:57.427372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f012af-60e0-40b0-8e54-93fcb9839446 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:57.428803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:57.431192 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f012af-60e0-40b0-8e54-93fcb9839446 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:57.444311 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51cfd1-a7f2-4c81-bbb7-665bb77f1c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:57.445248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51cfd1-a7f2-4c81-bbb7-665bb77f1c34 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:57.445735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51cfd1-a7f2-4c81-bbb7-665bb77f1c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:57.446187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51cfd1-a7f2-4c81-bbb7-665bb77f1c34 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:57.447162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51cfd1-a7f2-4c81-bbb7-665bb77f1c34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:57.448307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51cfd1-a7f2-4c81-bbb7-665bb77f1c34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:57.449971 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51cfd1-a7f2-4c81-bbb7-665bb77f1c34 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:57.467332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 313/631] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:00:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:57.468547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51cfd1-a7f2-4c81-bbb7-665bb77f1c34 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:57.601635 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 319/632] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:57 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:57.602857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f012af-60e0-40b0-8e54-93fcb9839446 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:57.604761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51cfd1-a7f2-4c81-bbb7-665bb77f1c34 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:57.634954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:57.642808 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a094e-86b9-4703-9c03-f0c7095612b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:57.643505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a094e-86b9-4703-9c03-f0c7095612b7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:57.643745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a094e-86b9-4703-9c03-f0c7095612b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:57.644021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a094e-86b9-4703-9c03-f0c7095612b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:57.655406 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a094e-86b9-4703-9c03-f0c7095612b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:57.656864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a094e-86b9-4703-9c03-f0c7095612b7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-631a094e-86b9-4703-9c03-f0c7095612b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:57.673064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:57.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:57.674972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:57.674972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:57.674972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:57.674972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:57.678926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 314/633] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:57.679811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a094e-86b9-4703-9c03-f0c7095612b7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:57.716745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:57.727686 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f012af-60e0-40b0-8e54-93fcb9839446 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:57.765936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7adff6-3df8-4271-b9e9-66f1c766e5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:57.766615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7adff6-3df8-4271-b9e9-66f1c766e5f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:57.766956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7adff6-3df8-4271-b9e9-66f1c766e5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:57.767195 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7adff6-3df8-4271-b9e9-66f1c766e5f8 None admin] PATH_INFO: `/v3/projects/d6e147b6607b481591ceb250f7208c16/users/41a399a3cc7446c6a8dad94075851797/roles/70e0c42f1bdd49178db398637f021bbc` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:57.783534 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7adff6-3df8-4271-b9e9-66f1c766e5f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6e147b6607b481591ceb250f7208c16, user_id=41a399a3cc7446c6a8dad94075851797, role_id=70e0c42f1bdd49178db398637f021bbc)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:57.784984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7adff6-3df8-4271-b9e9-66f1c766e5f8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:57.789812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a094e-86b9-4703-9c03-f0c7095612b7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:57.801161 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 320/634] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:00:57 2022] PUT /identity/v3/projects/d6e147b6607b481591ceb250f7208c16/users/41a399a3cc7446c6a8dad94075851797/roles/70e0c42f1bdd49178db398637f021bbc => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:00:57.802217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7adff6-3df8-4271-b9e9-66f1c766e5f8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:57.828700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b299b85-a874-4994-ae5e-5a765125c360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:57.829320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b299b85-a874-4994-ae5e-5a765125c360 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:57.829600 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b299b85-a874-4994-ae5e-5a765125c360 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:57.829870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b299b85-a874-4994-ae5e-5a765125c360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:57.843312 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b299b85-a874-4994-ae5e-5a765125c360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:57.844842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b299b85-a874-4994-ae5e-5a765125c360 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1b299b85-a874-4994-ae5e-5a765125c360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:57.861943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:57.863037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:57.863967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:57.863967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:57.863967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:57.863967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:57.863967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 315/635] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:57.864786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8373d1-68ad-4bcd-8526-dfe8c64fe563 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:57.864786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8373d1-68ad-4bcd-8526-dfe8c64fe563 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:57.865025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8373d1-68ad-4bcd-8526-dfe8c64fe563 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:57.870735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 316/636] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:57.870735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ce02b-eddc-40c1-8cd2-75f60a3c4eb5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:57.870735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ce02b-eddc-40c1-8cd2-75f60a3c4eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:57.870735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ce02b-eddc-40c1-8cd2-75f60a3c4eb5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:57.870735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 317/637] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:00:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:00:57.875046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ce02b-eddc-40c1-8cd2-75f60a3c4eb5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:57.919847 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:57.946347 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7adff6-3df8-4271-b9e9-66f1c766e5f8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:57.984226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:57.991128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811f834-717c-482d-9287-ccdbc7a186c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:57.992250 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811f834-717c-482d-9287-ccdbc7a186c5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:57.992534 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811f834-717c-482d-9287-ccdbc7a186c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:57.992803 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811f834-717c-482d-9287-ccdbc7a186c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:57.993497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811f834-717c-482d-9287-ccdbc7a186c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:57.993798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811f834-717c-482d-9287-ccdbc7a186c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:57.995366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811f834-717c-482d-9287-ccdbc7a186c5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:58.003289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ce02b-eddc-40c1-8cd2-75f60a3c4eb5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:58.007932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 321/638] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:58.012509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12942bee-859a-4dc1-92f3-eb5a0f46d933 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:58.012509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12942bee-859a-4dc1-92f3-eb5a0f46d933 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:58.012509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12942bee-859a-4dc1-92f3-eb5a0f46d933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:58.028114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-12942bee-859a-4dc1-92f3-eb5a0f46d933 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:58.029330 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-12942bee-859a-4dc1-92f3-eb5a0f46d933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:58.044627 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faa664-a37c-4be9-9213-9f34e7c50913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:58.045253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faa664-a37c-4be9-9213-9f34e7c50913 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:58.045534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faa664-a37c-4be9-9213-9f34e7c50913 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:58.045805 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faa664-a37c-4be9-9213-9f34e7c50913 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:58.046503 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58faa664-a37c-4be9-9213-9f34e7c50913 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:58.046849 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58faa664-a37c-4be9-9213-9f34e7c50913 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:58.048448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58faa664-a37c-4be9-9213-9f34e7c50913 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:58.066926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 318/639] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:58.066926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003300f1-fcbf-462c-bf7b-18cc3fd5eb25 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:58.066926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003300f1-fcbf-462c-bf7b-18cc3fd5eb25 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:58.066926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003300f1-fcbf-462c-bf7b-18cc3fd5eb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:58.083792 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-003300f1-fcbf-462c-bf7b-18cc3fd5eb25 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:00:58.085014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-003300f1-fcbf-462c-bf7b-18cc3fd5eb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:00:58.119126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 322/640] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:58.123593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12942bee-859a-4dc1-92f3-eb5a0f46d933 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:58.170112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 319/641] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:00:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:00:58.171396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003300f1-fcbf-462c-bf7b-18cc3fd5eb25 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:58.208180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:58.249897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12942bee-859a-4dc1-92f3-eb5a0f46d933 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:58.268351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:58.290924 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b958d-9ad6-4930-8162-ad7f3d134b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:58.291567 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b958d-9ad6-4930-8162-ad7f3d134b1f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:58.291860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b958d-9ad6-4930-8162-ad7f3d134b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:58.292139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b958d-9ad6-4930-8162-ad7f3d134b1f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:58.298466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003300f1-fcbf-462c-bf7b-18cc3fd5eb25 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:58.300333 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b958d-9ad6-4930-8162-ad7f3d134b1f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:58.301796 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b958d-9ad6-4930-8162-ad7f3d134b1f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-484b958d-9ad6-4930-8162-ad7f3d134b1f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:00:58.314298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:58.315317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:58.316274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:58.316274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:58.316274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:00:58.316274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:00:58.316274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:00:58.316274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:00:58.316274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:00:58.316274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:00:58.316274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:00:58.316274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:58.316274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 323/642] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:58.316274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b958d-9ad6-4930-8162-ad7f3d134b1f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:58.344123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29c283-0848-4f7b-ab4e-e7ae9cbbbe2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:58.344781 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29c283-0848-4f7b-ab4e-e7ae9cbbbe2a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:58.345065 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29c283-0848-4f7b-ab4e-e7ae9cbbbe2a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:58.345334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29c283-0848-4f7b-ab4e-e7ae9cbbbe2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:58.352570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29c283-0848-4f7b-ab4e-e7ae9cbbbe2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:58.354017 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29c283-0848-4f7b-ab4e-e7ae9cbbbe2a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-cc29c283-0848-4f7b-ab4e-e7ae9cbbbe2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:00:58.365695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:58.371989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:58.373155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:00:58.373155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:00:58.373155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:00:58.373155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:00:58.373155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:00:58.373155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:00:58.373155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:00:58.373155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:58.373155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 320/643] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:58.373155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29c283-0848-4f7b-ab4e-e7ae9cbbbe2a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:58.447741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b958d-9ad6-4930-8162-ad7f3d134b1f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:58.476186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:58.488898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29c283-0848-4f7b-ab4e-e7ae9cbbbe2a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:58.489341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e809360-6c80-493f-96a2-723ef71068c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:58.490554 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e809360-6c80-493f-96a2-723ef71068c0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:58.491069 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e809360-6c80-493f-96a2-723ef71068c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:58.491552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e809360-6c80-493f-96a2-723ef71068c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:58.503651 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e809360-6c80-493f-96a2-723ef71068c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-4e809360-6c80-493f-96a2-723ef71068c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:58.509605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:58.510589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:58.511552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:58.511552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:58.511552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:58.511552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:58.511552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:58.511552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:58.511552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:58.511552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:58.511552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:58.511552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:58.511552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:58.511552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:58.511552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 324/644] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:58.512362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e809360-6c80-493f-96a2-723ef71068c0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:58.538041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da214e6-53be-4690-b512-fc8952f17579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:58.538041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da214e6-53be-4690-b512-fc8952f17579 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:58.538041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da214e6-53be-4690-b512-fc8952f17579 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:58.538442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da214e6-53be-4690-b512-fc8952f17579 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:58.543391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:58.553822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da214e6-53be-4690-b512-fc8952f17579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6da214e6-53be-4690-b512-fc8952f17579 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:00:58.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:58.560052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:58.561098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:58.561098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:58.561098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:58.561098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:58.561098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:00:58.561098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:00:58.561098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:00:58.561098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:00:58.561098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:00:58.561098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:00:58.561098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:00:58.561098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:58.562263 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 321/645] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:58.562964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da214e6-53be-4690-b512-fc8952f17579 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:58.642987 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e809360-6c80-493f-96a2-723ef71068c0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:58.683516 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac857d45-fd5c-46ed-b6d6-a71f3cf124b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:58.684141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac857d45-fd5c-46ed-b6d6-a71f3cf124b2 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:58.684423 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac857d45-fd5c-46ed-b6d6-a71f3cf124b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:58.684700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac857d45-fd5c-46ed-b6d6-a71f3cf124b2 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:58.685495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac857d45-fd5c-46ed-b6d6-a71f3cf124b2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:58.685807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac857d45-fd5c-46ed-b6d6-a71f3cf124b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:58.686900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da214e6-53be-4690-b512-fc8952f17579 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:58.687358 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac857d45-fd5c-46ed-b6d6-a71f3cf124b2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:58.705526 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 325/646] 10.222.0.100 () {56 vars in 1142 bytes} [Mon Aug 1 15:00:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:58.710836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac857d45-fd5c-46ed-b6d6-a71f3cf124b2 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:58.729764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0dabc2-5df3-41b2-957d-b27b71920926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:58.730389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dabc2-5df3-41b2-957d-b27b71920926 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:58.730698 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dabc2-5df3-41b2-957d-b27b71920926 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:58.730958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dabc2-5df3-41b2-957d-b27b71920926 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:58.731747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0dabc2-5df3-41b2-957d-b27b71920926 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:58.732075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0dabc2-5df3-41b2-957d-b27b71920926 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:58.733497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0dabc2-5df3-41b2-957d-b27b71920926 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:58.751876 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 322/647] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:00:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:58.751876 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0dabc2-5df3-41b2-957d-b27b71920926 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:58.757476 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:58.813814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac857d45-fd5c-46ed-b6d6-a71f3cf124b2 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:58.815782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:58.849137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701135e-ac4a-44ce-8592-c9d5715880ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:58.850308 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701135e-ac4a-44ce-8592-c9d5715880ba None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:58.850593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701135e-ac4a-44ce-8592-c9d5715880ba None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:58.850897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701135e-ac4a-44ce-8592-c9d5715880ba None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:58.851613 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701135e-ac4a-44ce-8592-c9d5715880ba None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:58.851926 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701135e-ac4a-44ce-8592-c9d5715880ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:58.853294 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701135e-ac4a-44ce-8592-c9d5715880ba None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:58.865742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 326/648] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:00:58 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:58.865742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701135e-ac4a-44ce-8592-c9d5715880ba None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:58.868545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0dabc2-5df3-41b2-957d-b27b71920926 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:58.905248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d54f7-194d-4d92-8a4c-497c0d802364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:58.905787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d54f7-194d-4d92-8a4c-497c0d802364 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:58.906136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d54f7-194d-4d92-8a4c-497c0d802364 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:58.906412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d54f7-194d-4d92-8a4c-497c0d802364 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:58.907167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d54f7-194d-4d92-8a4c-497c0d802364 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:58.907475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d54f7-194d-4d92-8a4c-497c0d802364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:58.908878 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d54f7-194d-4d92-8a4c-497c0d802364 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:58.922467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 323/649] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:00:58 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:58.922467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d54f7-194d-4d92-8a4c-497c0d802364 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:58.978963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701135e-ac4a-44ce-8592-c9d5715880ba None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:58.988091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:59.016805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d9a6a-768b-40d7-bba4-83ca0138e3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:59.017649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d9a6a-768b-40d7-bba4-83ca0138e3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:59.018085 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d9a6a-768b-40d7-bba4-83ca0138e3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:59.018355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d9a6a-768b-40d7-bba4-83ca0138e3b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:59.036955 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d54f7-194d-4d92-8a4c-497c0d802364 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:59.054430 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:59.071753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef18139-206f-4f60-be67-d7166805583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:59.072374 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef18139-206f-4f60-be67-d7166805583f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:59.072661 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef18139-206f-4f60-be67-d7166805583f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:59.072939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef18139-206f-4f60-be67-d7166805583f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:59.073729 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef18139-206f-4f60-be67-d7166805583f None admin] RBAC: Adding query filter params (role.id=70e0c42f1bdd49178db398637f021bbc, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=ec50ac7c54324884a1b2a6c63d617907) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:59.074050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef18139-206f-4f60-be67-d7166805583f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:59.075748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef18139-206f-4f60-be67-d7166805583f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:59.082317 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 324/650] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:00:58 2022] GET /identity/v3/role_assignments?user.id=ec50ac7c54324884a1b2a6c63d617907&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=70e0c42f1bdd49178db398637f021bbc => generated 610 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:59.083138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef18139-206f-4f60-be67-d7166805583f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:59.111972 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d9a6a-768b-40d7-bba4-83ca0138e3b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:59.115082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d9a6a-768b-40d7-bba4-83ca0138e3b3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:59.216980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef18139-206f-4f60-be67-d7166805583f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:59.258013 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 327/651] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:00:58 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:00:59.259154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d9a6a-768b-40d7-bba4-83ca0138e3b3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:59.269213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed4e67-97fb-4510-8c78-0a5b9b4e21fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:59.270134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed4e67-97fb-4510-8c78-0a5b9b4e21fd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:59.270566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed4e67-97fb-4510-8c78-0a5b9b4e21fd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:59.271546 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed4e67-97fb-4510-8c78-0a5b9b4e21fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:59.274579 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:59.287382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed4e67-97fb-4510-8c78-0a5b9b4e21fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:59.289717 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaed4e67-97fb-4510-8c78-0a5b9b4e21fd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-eaed4e67-97fb-4510-8c78-0a5b9b4e21fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:59.322417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:59.323571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:59.324686 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:59.324686 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:59.324686 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:59.324686 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:59.324960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 325/652] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:59.326181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed4e67-97fb-4510-8c78-0a5b9b4e21fd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:59.372720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:59.383976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d9a6a-768b-40d7-bba4-83ca0138e3b3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:59.437113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b1138-641b-41d4-a94a-b450630b4937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:59.438306 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b1138-641b-41d4-a94a-b450630b4937 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:59.438872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b1138-641b-41d4-a94a-b450630b4937 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:59.439550 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b1138-641b-41d4-a94a-b450630b4937 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:59.449772 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaed4e67-97fb-4510-8c78-0a5b9b4e21fd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:59.455026 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b1138-641b-41d4-a94a-b450630b4937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:59.458630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b1138-641b-41d4-a94a-b450630b4937 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-4b7b1138-641b-41d4-a94a-b450630b4937 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:59.482628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:59.483777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:59.484776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:59.484776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:59.484776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:59.484776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:00:59.484776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 328/653] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:59.489430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b1138-641b-41d4-a94a-b450630b4937 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:59.496075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463f3ba-1d35-4e92-b4b5-d209a9e495b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:59.496970 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463f3ba-1d35-4e92-b4b5-d209a9e495b0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:59.497739 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463f3ba-1d35-4e92-b4b5-d209a9e495b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:59.498219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463f3ba-1d35-4e92-b4b5-d209a9e495b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:59.513268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463f3ba-1d35-4e92-b4b5-d209a9e495b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:59.515125 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463f3ba-1d35-4e92-b4b5-d209a9e495b0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6463f3ba-1d35-4e92-b4b5-d209a9e495b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:00:59.535736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:59.536912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:59.539127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:00:59.539127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:00:59.539127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:00:59.539127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:59.548636 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 326/654] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:59.548636 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463f3ba-1d35-4e92-b4b5-d209a9e495b0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:59.571104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:59.622940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b1138-641b-41d4-a94a-b450630b4937 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:59.668138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:59.676091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3078f-21b3-4018-be26-81ddd5e7b7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:59.676808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3078f-21b3-4018-be26-81ddd5e7b7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:59.677112 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3078f-21b3-4018-be26-81ddd5e7b7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:59.677644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d3078f-21b3-4018-be26-81ddd5e7b7d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:59.678168 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3078f-21b3-4018-be26-81ddd5e7b7d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:59.678479 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3078f-21b3-4018-be26-81ddd5e7b7d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:59.680092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d3078f-21b3-4018-be26-81ddd5e7b7d6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:59.687785 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463f3ba-1d35-4e92-b4b5-d209a9e495b0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:59.698548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 329/655] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:59.698548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3078f-21b3-4018-be26-81ddd5e7b7d6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:59.741432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273361e-8b91-487a-b5f1-a49a0f0dfc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:59.742002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273361e-8b91-487a-b5f1-a49a0f0dfc5b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:59.742299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273361e-8b91-487a-b5f1-a49a0f0dfc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:59.742968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273361e-8b91-487a-b5f1-a49a0f0dfc5b None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:59.743682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273361e-8b91-487a-b5f1-a49a0f0dfc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:59.743993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273361e-8b91-487a-b5f1-a49a0f0dfc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:59.746017 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0273361e-8b91-487a-b5f1-a49a0f0dfc5b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:59.762815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 327/656] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:59.764044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273361e-8b91-487a-b5f1-a49a0f0dfc5b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:59.815833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d3078f-21b3-4018-be26-81ddd5e7b7d6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:59.838867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:59.853861 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e582f-24b3-4a79-82be-fb4492876854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:59.854489 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e582f-24b3-4a79-82be-fb4492876854 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:59.855484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e582f-24b3-4a79-82be-fb4492876854 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:59.855484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e582f-24b3-4a79-82be-fb4492876854 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:59.855792 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e582f-24b3-4a79-82be-fb4492876854 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:00:59.856047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e582f-24b3-4a79-82be-fb4492876854 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:59.857426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e582f-24b3-4a79-82be-fb4492876854 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:59.874253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 330/657] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:00:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:00:59.874253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e582f-24b3-4a79-82be-fb4492876854 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:59.894092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0273361e-8b91-487a-b5f1-a49a0f0dfc5b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:00:59.933565 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aad111-2687-4130-954f-50cdcf442c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:00:59.934219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aad111-2687-4130-954f-50cdcf442c3d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:00:59.934505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aad111-2687-4130-954f-50cdcf442c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:00:59.934820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aad111-2687-4130-954f-50cdcf442c3d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:00:59.943931 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aad111-2687-4130-954f-50cdcf442c3d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:00:59.945396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aad111-2687-4130-954f-50cdcf442c3d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-85aad111-2687-4130-954f-50cdcf442c3d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:00:59.957923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:00:59.959579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:00:59.960534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:00:59.960534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:00:59.960534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:00:59.960534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:00:59.960534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:00:59.960534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:00:59.960534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:00:59.960534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:00:59.960534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 15:00:59.960534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:00:59.960534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 328/658] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:00:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:00:59.960534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aad111-2687-4130-954f-50cdcf442c3d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:00:59.973348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:00:59.984025 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e582f-24b3-4a79-82be-fb4492876854 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:00.022248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c3780-ceb8-40c3-84c1-485e800d93ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:00.022844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c3780-ceb8-40c3-84c1-485e800d93ed None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:00.023130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c3780-ceb8-40c3-84c1-485e800d93ed None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:00.023405 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c3780-ceb8-40c3-84c1-485e800d93ed None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:00.031573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c3780-ceb8-40c3-84c1-485e800d93ed None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:00.033028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c3780-ceb8-40c3-84c1-485e800d93ed None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-d69c3780-ceb8-40c3-84c1-485e800d93ed None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:00.045066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:00.046497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:00.047484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:00.047484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:00.047484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:00.047484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:00.047484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:00.047484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:00.047484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:00.047484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:00.047484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 15:01:00.047484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:00.051022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 331/659] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:00:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:00.052109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b002-0a80-4919-bf48-6484bb5c165a None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:00.052400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b002-0a80-4919-bf48-6484bb5c165a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:00.052710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805b002-0a80-4919-bf48-6484bb5c165a None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:00.054113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 332/660] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:00.054703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805b002-0a80-4919-bf48-6484bb5c165a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:00.080375 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aad111-2687-4130-954f-50cdcf442c3d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:00.102097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:00.127601 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f010d-d99d-458f-be46-b8e22f04766f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:00.128248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f010d-d99d-458f-be46-b8e22f04766f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:00.128529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f010d-d99d-458f-be46-b8e22f04766f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:00.128801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f010d-d99d-458f-be46-b8e22f04766f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:00.136778 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f010d-d99d-458f-be46-b8e22f04766f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:00.138233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f010d-d99d-458f-be46-b8e22f04766f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-153f010d-d99d-458f-be46-b8e22f04766f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:00.150811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:00.152155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:00.153058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:00.153058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:00.153058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:00.153058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:00.153058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:00.153058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:00.153058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:00.153058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:00.153058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 329/661] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:00:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:00.153058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f010d-d99d-458f-be46-b8e22f04766f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:00.177653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805b002-0a80-4919-bf48-6484bb5c165a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:00.219849 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92042a-e566-4677-b9cb-65ba7ea75e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:00.220743 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92042a-e566-4677-b9cb-65ba7ea75e81 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:00.221032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92042a-e566-4677-b9cb-65ba7ea75e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:00.221310 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92042a-e566-4677-b9cb-65ba7ea75e81 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:00.222109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92042a-e566-4677-b9cb-65ba7ea75e81 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:00.222424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92042a-e566-4677-b9cb-65ba7ea75e81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:00.223214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:00.226224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92042a-e566-4677-b9cb-65ba7ea75e81 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:00.247180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 333/662] 10.222.0.100 () {56 vars in 1142 bytes} [Mon Aug 1 15:01:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:00.248789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf8db5-ad3b-4136-8299-a5dcc1acf4eb None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:00.249292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf8db5-ad3b-4136-8299-a5dcc1acf4eb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:00.249637 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf8db5-ad3b-4136-8299-a5dcc1acf4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:00.265557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f010d-d99d-458f-be46-b8e22f04766f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:00.268048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d3cf8db5-ad3b-4136-8299-a5dcc1acf4eb None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:00.269241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d3cf8db5-ad3b-4136-8299-a5dcc1acf4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:00.303664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df16cf2-a964-4bcd-8ae5-feaa610b26f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:00.304215 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df16cf2-a964-4bcd-8ae5-feaa610b26f4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:00.304498 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df16cf2-a964-4bcd-8ae5-feaa610b26f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:00.304774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df16cf2-a964-4bcd-8ae5-feaa610b26f4 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:00.305572 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df16cf2-a964-4bcd-8ae5-feaa610b26f4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:00.305882 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df16cf2-a964-4bcd-8ae5-feaa610b26f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:00.307359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df16cf2-a964-4bcd-8ae5-feaa610b26f4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:00.326649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 330/663] 10.222.0.100 () {56 vars in 1144 bytes} [Mon Aug 1 15:01:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:00.327515 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df16cf2-a964-4bcd-8ae5-feaa610b26f4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:00.340602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 334/664] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:01:00.341720 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf8db5-ad3b-4136-8299-a5dcc1acf4eb None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:00.372342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:00.453949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf8db5-ad3b-4136-8299-a5dcc1acf4eb None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:00.454161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df16cf2-a964-4bcd-8ae5-feaa610b26f4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:00.479563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:00.491796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53d355-1668-47e4-8a42-0b1c09fb1c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:00.491963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59140c4b-c5a9-46d0-9754-658d3f69bea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:00.492418 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59140c4b-c5a9-46d0-9754-658d3f69bea8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:00.492530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53d355-1668-47e4-8a42-0b1c09fb1c14 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:00.492701 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59140c4b-c5a9-46d0-9754-658d3f69bea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:00.492809 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53d355-1668-47e4-8a42-0b1c09fb1c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:00.492969 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59140c4b-c5a9-46d0-9754-658d3f69bea8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:00.493075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53d355-1668-47e4-8a42-0b1c09fb1c14 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:00.493870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53d355-1668-47e4-8a42-0b1c09fb1c14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:00.494138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53d355-1668-47e4-8a42-0b1c09fb1c14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:00.495642 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53d355-1668-47e4-8a42-0b1c09fb1c14 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:00.508370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 331/665] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:00.509024 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53d355-1668-47e4-8a42-0b1c09fb1c14 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:00.591829 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59140c4b-c5a9-46d0-9754-658d3f69bea8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:00.593401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59140c4b-c5a9-46d0-9754-658d3f69bea8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:00.620761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53d355-1668-47e4-8a42-0b1c09fb1c14 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:00.630198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:00.659085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd3b3c-cb24-47cc-b50a-5a418a08253e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:00.659830 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd3b3c-cb24-47cc-b50a-5a418a08253e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:00.660115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd3b3c-cb24-47cc-b50a-5a418a08253e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:00.660396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd3b3c-cb24-47cc-b50a-5a418a08253e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:00.709323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 335/666] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:00 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:01:00.709323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59140c4b-c5a9-46d0-9754-658d3f69bea8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:00.725594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:00.744673 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd3b3c-cb24-47cc-b50a-5a418a08253e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:00.746179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd3b3c-cb24-47cc-b50a-5a418a08253e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:00.825458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59140c4b-c5a9-46d0-9754-658d3f69bea8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:00.872161 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:00.874409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 332/667] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:00 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:01:00.875904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd3b3c-cb24-47cc-b50a-5a418a08253e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:00.888334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccd208-1a6f-4e81-abc7-e56765061e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:00.889740 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccd208-1a6f-4e81-abc7-e56765061e59 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:00.890228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccd208-1a6f-4e81-abc7-e56765061e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:00.891035 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ccd208-1a6f-4e81-abc7-e56765061e59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:00.912993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ccd208-1a6f-4e81-abc7-e56765061e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-23ccd208-1a6f-4e81-abc7-e56765061e59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:00.917379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:00.918334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:00.919217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:00.919217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:00.919217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:00.919217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:00.919217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:00.919217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:00.919217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:00.919217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:00.919217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:00.919217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:00.919217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:01:00.919217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:00.919217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 336/668] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:01:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:00.919930 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccd208-1a6f-4e81-abc7-e56765061e59 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:00.984887 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:01.002638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd3b3c-cb24-47cc-b50a-5a418a08253e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:01.034047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ccd208-1a6f-4e81-abc7-e56765061e59 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:01.040551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a9090-71f0-4744-b520-3b259a4249d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:01.041180 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a9090-71f0-4744-b520-3b259a4249d8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:01.041461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a9090-71f0-4744-b520-3b259a4249d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:01.041735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a9090-71f0-4744-b520-3b259a4249d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:01.046034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a9090-71f0-4744-b520-3b259a4249d8 None admin] RBAC: Adding query filter params (role.id=70e0c42f1bdd49178db398637f021bbc, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=ead1b36d35f3485f89edd15b8443feea) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:01.046034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a9090-71f0-4744-b520-3b259a4249d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:01.046034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a9090-71f0-4744-b520-3b259a4249d8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:01.057363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 333/669] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:00 2022] GET /identity/v3/role_assignments?user.id=ead1b36d35f3485f89edd15b8443feea&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=70e0c42f1bdd49178db398637f021bbc => generated 610 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:01.067807 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a9090-71f0-4744-b520-3b259a4249d8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:01.096174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af277eca-c561-4ac3-9eba-016aabb7c89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:01.097306 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af277eca-c561-4ac3-9eba-016aabb7c89f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:01.097794 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af277eca-c561-4ac3-9eba-016aabb7c89f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:01.098240 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af277eca-c561-4ac3-9eba-016aabb7c89f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:01.117837 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af277eca-c561-4ac3-9eba-016aabb7c89f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:01.119234 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af277eca-c561-4ac3-9eba-016aabb7c89f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-af277eca-c561-4ac3-9eba-016aabb7c89f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:01.136980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:01.137911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:01.139509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:01.139509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:01.139509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:01.139509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:01.139509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 337/670] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:01.139509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af277eca-c561-4ac3-9eba-016aabb7c89f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:01.191877 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:01.231542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a9090-71f0-4744-b520-3b259a4249d8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:01.277704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af277eca-c561-4ac3-9eba-016aabb7c89f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:01.280893 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75dc45-cd5f-418d-b6e8-e21dd209cad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:01.281766 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75dc45-cd5f-418d-b6e8-e21dd209cad7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:01.282230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75dc45-cd5f-418d-b6e8-e21dd209cad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:01.282738 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75dc45-cd5f-418d-b6e8-e21dd209cad7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:01.299074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75dc45-cd5f-418d-b6e8-e21dd209cad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:01.300942 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75dc45-cd5f-418d-b6e8-e21dd209cad7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:01.311427 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:01.317527 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819a4f3-054c-4ce5-a8e8-196723b5982d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:01.318230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819a4f3-054c-4ce5-a8e8-196723b5982d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:01.319095 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819a4f3-054c-4ce5-a8e8-196723b5982d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:01.319368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819a4f3-054c-4ce5-a8e8-196723b5982d None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7a75dc45-cd5f-418d-b6e8-e21dd209cad7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:01.319722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:01.320763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:01.321859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:01.321859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:01.321859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:01.321859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:01.322093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819a4f3-054c-4ce5-a8e8-196723b5982d None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:01.322093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819a4f3-054c-4ce5-a8e8-196723b5982d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:01.322093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819a4f3-054c-4ce5-a8e8-196723b5982d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:01.324410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 334/671] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:01.325685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75dc45-cd5f-418d-b6e8-e21dd209cad7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:01.332578 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 338/672] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:01 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:01.334045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc295b-57a4-44aa-932a-2127b20b4935 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:01.334473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc295b-57a4-44aa-932a-2127b20b4935 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:01.334927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc295b-57a4-44aa-932a-2127b20b4935 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:01.336579 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 339/673] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:01.337562 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc295b-57a4-44aa-932a-2127b20b4935 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:01.454439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc295b-57a4-44aa-932a-2127b20b4935 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:01.456286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:01.465949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75dc45-cd5f-418d-b6e8-e21dd209cad7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:01.498140 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef94347-4df3-4687-85e7-6d586965e4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:01.498792 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef94347-4df3-4687-85e7-6d586965e4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:01.499113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef94347-4df3-4687-85e7-6d586965e4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:01.499391 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef94347-4df3-4687-85e7-6d586965e4c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:01.500085 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef94347-4df3-4687-85e7-6d586965e4c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:01.500387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef94347-4df3-4687-85e7-6d586965e4c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:01.501748 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef94347-4df3-4687-85e7-6d586965e4c5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:01.507443 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d9accd-ffa1-48f9-aec7-80e431e8c8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:01.508065 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9accd-ffa1-48f9-aec7-80e431e8c8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:01.508351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9accd-ffa1-48f9-aec7-80e431e8c8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:01.508630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d9accd-ffa1-48f9-aec7-80e431e8c8f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:01.509318 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9accd-ffa1-48f9-aec7-80e431e8c8f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:01.509626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9accd-ffa1-48f9-aec7-80e431e8c8f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:01.511015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d9accd-ffa1-48f9-aec7-80e431e8c8f1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:01.514438 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 340/674] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:01.515065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef94347-4df3-4687-85e7-6d586965e4c5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:01.532225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 335/675] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:01.532225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b5657-6426-4875-8182-40f53f204ee0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:01.532225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b5657-6426-4875-8182-40f53f204ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:01.532225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b5657-6426-4875-8182-40f53f204ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:01.554699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cc7b5657-6426-4875-8182-40f53f204ee0 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:01.556020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cc7b5657-6426-4875-8182-40f53f204ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:01.621957 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:01.641019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef94347-4df3-4687-85e7-6d586965e4c5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:01.641214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 336/676] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:01:01.641214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b5657-6426-4875-8182-40f53f204ee0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:01.681314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8d61b-d1fe-4078-991a-26f247a0ceeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:01.681969 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8d61b-d1fe-4078-991a-26f247a0ceeb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:01.682289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8d61b-d1fe-4078-991a-26f247a0ceeb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:01.682566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8d61b-d1fe-4078-991a-26f247a0ceeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:01.693516 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8d61b-d1fe-4078-991a-26f247a0ceeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:01.694996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8d61b-d1fe-4078-991a-26f247a0ceeb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-d9c8d61b-d1fe-4078-991a-26f247a0ceeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:01.711094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:01.711984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:01.713163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:01.713163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:01.713163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:01.713163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:01.713163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 341/677] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:01.713448 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8d61b-d1fe-4078-991a-26f247a0ceeb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:01.737232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:01.757954 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b5657-6426-4875-8182-40f53f204ee0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:01.794858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8403e74-451d-4c08-8a5a-34366e85ae1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:01.795523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8403e74-451d-4c08-8a5a-34366e85ae1b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:01.795802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8403e74-451d-4c08-8a5a-34366e85ae1b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:01.796078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8403e74-451d-4c08-8a5a-34366e85ae1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:01.811016 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8403e74-451d-4c08-8a5a-34366e85ae1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:01.812857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8403e74-451d-4c08-8a5a-34366e85ae1b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f8403e74-451d-4c08-8a5a-34366e85ae1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:01.825364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:01.826347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:01.827415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:01.827415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:01.827415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:01.827415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:01.827415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:01.827415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:01.827415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:01.827415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:01.831702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 337/678] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:01.832738 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8403e74-451d-4c08-8a5a-34366e85ae1b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:01.840096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8d61b-d1fe-4078-991a-26f247a0ceeb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:01.881186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf80f4-cb31-4b8b-bc40-9d91dbadc9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:01.881832 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf80f4-cb31-4b8b-bc40-9d91dbadc9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:01.882629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf80f4-cb31-4b8b-bc40-9d91dbadc9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:01.882933 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf80f4-cb31-4b8b-bc40-9d91dbadc9b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:01.890888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf80f4-cb31-4b8b-bc40-9d91dbadc9b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:01.892366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf80f4-cb31-4b8b-bc40-9d91dbadc9b4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:01.894447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-91bf80f4-cb31-4b8b-bc40-9d91dbadc9b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:01.905385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:01.906481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:01.907676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:01.907676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:01.907676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:01.907676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:01.907676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:01.907676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:01.907676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:01.907676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:01.909495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 342/679] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:01.910440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf80f4-cb31-4b8b-bc40-9d91dbadc9b4 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:01.964989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8403e74-451d-4c08-8a5a-34366e85ae1b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:02.005946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:02.012653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980df54-ac00-4ac5-b7fc-c653f00ae495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:02.014263 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980df54-ac00-4ac5-b7fc-c653f00ae495 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:02.014754 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980df54-ac00-4ac5-b7fc-c653f00ae495 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:02.015393 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980df54-ac00-4ac5-b7fc-c653f00ae495 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:02.026394 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf80f4-cb31-4b8b-bc40-9d91dbadc9b4 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:02.032647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980df54-ac00-4ac5-b7fc-c653f00ae495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c980df54-ac00-4ac5-b7fc-c653f00ae495 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:02.038944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:02.039883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:02.040745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:02.040745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:02.040745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:02.040745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:02.040745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:02.040745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:02.040745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:02.040745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:02.040745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:02.040745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:02.040745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:02.040745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:02.040745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 338/680] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:01:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:02.041455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980df54-ac00-4ac5-b7fc-c653f00ae495 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:02.067161 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c6848-d37f-4f7d-b507-a5e02eab41b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:02.068755 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c6848-d37f-4f7d-b507-a5e02eab41b6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:02.069184 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c6848-d37f-4f7d-b507-a5e02eab41b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:02.069584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c6848-d37f-4f7d-b507-a5e02eab41b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:02.070277 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c6848-d37f-4f7d-b507-a5e02eab41b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:02.070763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c6848-d37f-4f7d-b507-a5e02eab41b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:02.072795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c6848-d37f-4f7d-b507-a5e02eab41b6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:02.086471 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 343/681] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:02.087344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c6848-d37f-4f7d-b507-a5e02eab41b6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:02.168052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980df54-ac00-4ac5-b7fc-c653f00ae495 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:02.180217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:02.207531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c6848-d37f-4f7d-b507-a5e02eab41b6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:02.220594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5750d77-8d4c-4349-82ec-b02bbca524ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:02.221220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5750d77-8d4c-4349-82ec-b02bbca524ce None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:02.221375 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5750d77-8d4c-4349-82ec-b02bbca524ce None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:02.221659 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5750d77-8d4c-4349-82ec-b02bbca524ce None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:02.222769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5750d77-8d4c-4349-82ec-b02bbca524ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:02.223014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5750d77-8d4c-4349-82ec-b02bbca524ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:02.224430 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5750d77-8d4c-4349-82ec-b02bbca524ce None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:02.238902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 339/682] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:02.239805 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5750d77-8d4c-4349-82ec-b02bbca524ce None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:02.264381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d87d4a-4401-4348-9652-4a17220661f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:02.265009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d87d4a-4401-4348-9652-4a17220661f9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:02.265844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d87d4a-4401-4348-9652-4a17220661f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:02.265844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d87d4a-4401-4348-9652-4a17220661f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:02.266316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d87d4a-4401-4348-9652-4a17220661f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:02.266622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d87d4a-4401-4348-9652-4a17220661f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:02.268140 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d87d4a-4401-4348-9652-4a17220661f9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:02.270267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:02.290826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 344/683] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:02.290826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d87d4a-4401-4348-9652-4a17220661f9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:02.364325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5750d77-8d4c-4349-82ec-b02bbca524ce None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:02.401719 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b275de-5342-4cbb-ae4d-f575688ab8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:02.402343 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b275de-5342-4cbb-ae4d-f575688ab8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:02.402628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b275de-5342-4cbb-ae4d-f575688ab8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:02.402926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b275de-5342-4cbb-ae4d-f575688ab8b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:02.403653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b275de-5342-4cbb-ae4d-f575688ab8b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:02.403961 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b275de-5342-4cbb-ae4d-f575688ab8b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:02.405341 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b275de-5342-4cbb-ae4d-f575688ab8b2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:02.406201 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d87d4a-4401-4348-9652-4a17220661f9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:02.418214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 340/684] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:01:02 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:02.418214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b275de-5342-4cbb-ae4d-f575688ab8b2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:02.446043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a6e23-ff8f-48e9-a8a3-58d13751f573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:02.446698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a6e23-ff8f-48e9-a8a3-58d13751f573 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:02.446986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a6e23-ff8f-48e9-a8a3-58d13751f573 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:02.447262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a6e23-ff8f-48e9-a8a3-58d13751f573 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:02.451264 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:02.455823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a6e23-ff8f-48e9-a8a3-58d13751f573 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:02.457279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a6e23-ff8f-48e9-a8a3-58d13751f573 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-8b3a6e23-ff8f-48e9-a8a3-58d13751f573 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:02.470180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:02.471295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:02.472269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:02.472269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:02.472269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:02.472269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:02.472269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:02.472269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:02.472269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:02.472269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:02.472269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:01:02.472269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:02.472269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 345/685] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:01:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:02.472269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a6e23-ff8f-48e9-a8a3-58d13751f573 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:02.532373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:02.534713 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b275de-5342-4cbb-ae4d-f575688ab8b2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:02.577945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d41fef-69fe-40c2-8621-ab32346032b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:02.578593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d41fef-69fe-40c2-8621-ab32346032b5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:02.578907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d41fef-69fe-40c2-8621-ab32346032b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:02.579184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d41fef-69fe-40c2-8621-ab32346032b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:02.579982 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d41fef-69fe-40c2-8621-ab32346032b5 None admin] RBAC: Adding query filter params (role.id=70e0c42f1bdd49178db398637f021bbc, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=e7f757eabcb84de6ab042e0ac5a8158c) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:02.580289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d41fef-69fe-40c2-8621-ab32346032b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:02.581720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d41fef-69fe-40c2-8621-ab32346032b5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:02.588503 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a6e23-ff8f-48e9-a8a3-58d13751f573 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:02.598866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 341/686] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:02 2022] GET /identity/v3/role_assignments?user.id=e7f757eabcb84de6ab042e0ac5a8158c&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=70e0c42f1bdd49178db398637f021bbc => generated 610 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:02.598866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d41fef-69fe-40c2-8621-ab32346032b5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:02.639278 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49aa00-d32e-4aca-ac38-c45532fa247e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:02.640016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49aa00-d32e-4aca-ac38-c45532fa247e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:02.640363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49aa00-d32e-4aca-ac38-c45532fa247e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:02.640793 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49aa00-d32e-4aca-ac38-c45532fa247e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:02.641773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa49aa00-d32e-4aca-ac38-c45532fa247e None admin] RBAC: Adding query filter params (role.id=70e0c42f1bdd49178db398637f021bbc, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=e75edfc5bb404d63b27a355d5ab8af01) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:02.642149 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa49aa00-d32e-4aca-ac38-c45532fa247e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:02.643719 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa49aa00-d32e-4aca-ac38-c45532fa247e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:02.657812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 346/687] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:02 2022] GET /identity/v3/role_assignments?user.id=e75edfc5bb404d63b27a355d5ab8af01&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=70e0c42f1bdd49178db398637f021bbc => generated 610 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:02.657812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49aa00-d32e-4aca-ac38-c45532fa247e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:02.807517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:02.817720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d41fef-69fe-40c2-8621-ab32346032b5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:02.880085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05761eae-33c5-4123-b2bb-a6bd9cd3b17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:02.881347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05761eae-33c5-4123-b2bb-a6bd9cd3b17d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:02.881459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49aa00-d32e-4aca-ac38-c45532fa247e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:02.881624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05761eae-33c5-4123-b2bb-a6bd9cd3b17d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:02.881865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05761eae-33c5-4123-b2bb-a6bd9cd3b17d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:02.897215 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05761eae-33c5-4123-b2bb-a6bd9cd3b17d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:02.899164 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05761eae-33c5-4123-b2bb-a6bd9cd3b17d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:02.902473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-05761eae-33c5-4123-b2bb-a6bd9cd3b17d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:02.920939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:02.922045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:02.923008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:02.923008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:02.923008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:02.923008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:02.923008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 342/688] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:02.929329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05761eae-33c5-4123-b2bb-a6bd9cd3b17d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:02.934417 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edfe0e7-442d-45c1-b79d-5f5a82ee341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:02.935167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfe0e7-442d-45c1-b79d-5f5a82ee341f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:02.935450 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfe0e7-442d-45c1-b79d-5f5a82ee341f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:02.935913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfe0e7-442d-45c1-b79d-5f5a82ee341f None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:02.936604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edfe0e7-442d-45c1-b79d-5f5a82ee341f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:02.936914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edfe0e7-442d-45c1-b79d-5f5a82ee341f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:02.938363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edfe0e7-442d-45c1-b79d-5f5a82ee341f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:02.969580 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 347/689] 10.222.0.100 () {56 vars in 1140 bytes} [Mon Aug 1 15:01:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:02.971446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edfe0e7-442d-45c1-b79d-5f5a82ee341f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:03.067101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05761eae-33c5-4123-b2bb-a6bd9cd3b17d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:03.124813 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edfe0e7-442d-45c1-b79d-5f5a82ee341f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:03.125015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710d1be-ae72-4be7-9cfa-9d9cd4e743a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:03.125503 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710d1be-ae72-4be7-9cfa-9d9cd4e743a5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:03.125789 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710d1be-ae72-4be7-9cfa-9d9cd4e743a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:03.126067 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710d1be-ae72-4be7-9cfa-9d9cd4e743a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:03.126980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710d1be-ae72-4be7-9cfa-9d9cd4e743a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:03.127292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710d1be-ae72-4be7-9cfa-9d9cd4e743a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:03.128672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710d1be-ae72-4be7-9cfa-9d9cd4e743a5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:03.143949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 343/690] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:03.150078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710d1be-ae72-4be7-9cfa-9d9cd4e743a5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:03.162761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:03.188797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07093eb1-7cc1-4a3d-bde6-6ec59422481e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:03.189429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07093eb1-7cc1-4a3d-bde6-6ec59422481e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:03.189710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07093eb1-7cc1-4a3d-bde6-6ec59422481e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:03.189987 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07093eb1-7cc1-4a3d-bde6-6ec59422481e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:03.275623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:03.305470 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07093eb1-7cc1-4a3d-bde6-6ec59422481e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:03.306989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07093eb1-7cc1-4a3d-bde6-6ec59422481e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:03.307938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710d1be-ae72-4be7-9cfa-9d9cd4e743a5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:03.345776 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef4c38-f3c8-48b2-8e9a-e6dfabe7bdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:03.346860 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef4c38-f3c8-48b2-8e9a-e6dfabe7bdae None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:03.347362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef4c38-f3c8-48b2-8e9a-e6dfabe7bdae None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:03.347831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef4c38-f3c8-48b2-8e9a-e6dfabe7bdae None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:03.356929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef4c38-f3c8-48b2-8e9a-e6dfabe7bdae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:03.358397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef4c38-f3c8-48b2-8e9a-e6dfabe7bdae None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-deef4c38-f3c8-48b2-8e9a-e6dfabe7bdae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:03.374101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:03.375161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:03.376183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:03.376183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:03.376183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:03.376183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:03.376183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:03.376183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:03.376183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:03.376183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:03.376183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:01:03.376183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:03.376183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 344/691] 10.222.0.100 () {56 vars in 1098 bytes} [Mon Aug 1 15:01:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:03.376782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f25318-960b-4049-994d-613ee267060f None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:03.376782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f25318-960b-4049-994d-613ee267060f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:03.377450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f25318-960b-4049-994d-613ee267060f None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:03.379649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 345/692] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:03.382163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f25318-960b-4049-994d-613ee267060f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:03.435659 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 348/693] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:02 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:01:03.437169 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06b9d5-bd81-4659-a1b9-d9c678756f83 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:03.437553 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06b9d5-bd81-4659-a1b9-d9c678756f83 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:03.437932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06b9d5-bd81-4659-a1b9-d9c678756f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:03.462378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2e06b9d5-bd81-4659-a1b9-d9c678756f83 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:03.463943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:03.465888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2e06b9d5-bd81-4659-a1b9-d9c678756f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:03.506280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f25318-960b-4049-994d-613ee267060f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:03.553311 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067e6c8-f1b5-4fe2-84b7-9549ada5a52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:03.554088 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067e6c8-f1b5-4fe2-84b7-9549ada5a52b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:03.554472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067e6c8-f1b5-4fe2-84b7-9549ada5a52b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:03.554983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0067e6c8-f1b5-4fe2-84b7-9549ada5a52b None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:03.556005 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067e6c8-f1b5-4fe2-84b7-9549ada5a52b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:03.556439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067e6c8-f1b5-4fe2-84b7-9549ada5a52b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:03.557996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0067e6c8-f1b5-4fe2-84b7-9549ada5a52b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:03.560478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 349/694] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:01:03.561831 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06b9d5-bd81-4659-a1b9-d9c678756f83 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:03.571427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:03.582720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 346/695] 10.222.0.100 () {56 vars in 1138 bytes} [Mon Aug 1 15:01:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:03.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067e6c8-f1b5-4fe2-84b7-9549ada5a52b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:03.683665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06b9d5-bd81-4659-a1b9-d9c678756f83 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:03.723678 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56673471-bab4-441d-bda3-22041cfc1919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:03.724452 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56673471-bab4-441d-bda3-22041cfc1919 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:03.724975 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56673471-bab4-441d-bda3-22041cfc1919 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:03.725328 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56673471-bab4-441d-bda3-22041cfc1919 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:03.737521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56673471-bab4-441d-bda3-22041cfc1919 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:03.739181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56673471-bab4-441d-bda3-22041cfc1919 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:03.740391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0067e6c8-f1b5-4fe2-84b7-9549ada5a52b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-56673471-bab4-441d-bda3-22041cfc1919 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:03.757050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:03.758068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:03.759042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:03.759042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:03.759042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:03.759042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:03.762968 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 350/696] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:03.763946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56673471-bab4-441d-bda3-22041cfc1919 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:03.791197 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:03.801861 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aa2d0-475f-446e-9a71-89770f13ddfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:03.802529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa2d0-475f-446e-9a71-89770f13ddfb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:03.802848 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa2d0-475f-446e-9a71-89770f13ddfb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:03.803157 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa2d0-475f-446e-9a71-89770f13ddfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:03.842934 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:03.895545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29aa2d0-475f-446e-9a71-89770f13ddfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:03.898498 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29aa2d0-475f-446e-9a71-89770f13ddfb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:03.907924 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56673471-bab4-441d-bda3-22041cfc1919 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:03.946381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1655858-e673-41b9-86d9-887baac1d628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:03.946985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1655858-e673-41b9-86d9-887baac1d628 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:03.947270 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1655858-e673-41b9-86d9-887baac1d628 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:03.947548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1655858-e673-41b9-86d9-887baac1d628 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:03.948186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1655858-e673-41b9-86d9-887baac1d628 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:03.949560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1655858-e673-41b9-86d9-887baac1d628 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-e1655858-e673-41b9-86d9-887baac1d628 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:03.963538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:03.964572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:01:03.965467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:03.969788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 351/697] 10.222.0.100 () {56 vars in 1134 bytes} [Mon Aug 1 15:01:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:03.970835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1655858-e673-41b9-86d9-887baac1d628 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:04.015256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 347/698] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:03 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 15:01:04.016431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aa2d0-475f-446e-9a71-89770f13ddfb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:04.045970 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:04.091404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1655858-e673-41b9-86d9-887baac1d628 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:04.120207 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:04.134791 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16336392-bd7a-4c2d-966a-af020e9def66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:04.135441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16336392-bd7a-4c2d-966a-af020e9def66 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:04.135727 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16336392-bd7a-4c2d-966a-af020e9def66 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:04.136106 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16336392-bd7a-4c2d-966a-af020e9def66 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:04.136721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16336392-bd7a-4c2d-966a-af020e9def66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:04.137031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16336392-bd7a-4c2d-966a-af020e9def66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:04.138440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16336392-bd7a-4c2d-966a-af020e9def66 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:04.139210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aa2d0-475f-446e-9a71-89770f13ddfb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:04.156883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 352/699] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:04.157852 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16336392-bd7a-4c2d-966a-af020e9def66 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:04.178589 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53185c89-2edf-444e-9b6d-46cdcec77f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:04.179761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53185c89-2edf-444e-9b6d-46cdcec77f9b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:04.180247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53185c89-2edf-444e-9b6d-46cdcec77f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:04.180722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53185c89-2edf-444e-9b6d-46cdcec77f9b None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:04.181606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53185c89-2edf-444e-9b6d-46cdcec77f9b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:04.182111 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53185c89-2edf-444e-9b6d-46cdcec77f9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:04.183857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53185c89-2edf-444e-9b6d-46cdcec77f9b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:04.189723 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 348/700] 10.222.0.100 () {56 vars in 1129 bytes} [Mon Aug 1 15:01:04 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:04.190830 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53185c89-2edf-444e-9b6d-46cdcec77f9b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:04.279278 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16336392-bd7a-4c2d-966a-af020e9def66 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:04.291710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:04.302258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53185c89-2edf-444e-9b6d-46cdcec77f9b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:04.334798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814eb889-9566-4f9e-957d-869f391b70ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:04.335588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814eb889-9566-4f9e-957d-869f391b70ba None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:04.335888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814eb889-9566-4f9e-957d-869f391b70ba None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:04.336169 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814eb889-9566-4f9e-957d-869f391b70ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:04.342016 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89717b25-6f7a-4f1a-bc95-e9cd5a36bbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:04.342985 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89717b25-6f7a-4f1a-bc95-e9cd5a36bbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:04.343473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89717b25-6f7a-4f1a-bc95-e9cd5a36bbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:04.343945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89717b25-6f7a-4f1a-bc95-e9cd5a36bbf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:04.349511 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814eb889-9566-4f9e-957d-869f391b70ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:04.351396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814eb889-9566-4f9e-957d-869f391b70ba None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:04.352172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89717b25-6f7a-4f1a-bc95-e9cd5a36bbf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:04.354010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89717b25-6f7a-4f1a-bc95-e9cd5a36bbf3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-814eb889-9566-4f9e-957d-869f391b70ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:04.367225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:04.368139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:04.369039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:04.369039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:04.369039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:04.369039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-89717b25-6f7a-4f1a-bc95-e9cd5a36bbf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:04.369363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:04.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:04.371073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:04.371073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:04.371073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:04.371073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:04.371073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:04.371073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:04.371073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:04.371073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:04.378686 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 353/702] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:04.378686 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814eb889-9566-4f9e-957d-869f391b70ba None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:04.379061 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 349/702] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:04.379061 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89717b25-6f7a-4f1a-bc95-e9cd5a36bbf3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:04.415633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:04.483763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814eb889-9566-4f9e-957d-869f391b70ba None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:04.491776 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89717b25-6f7a-4f1a-bc95-e9cd5a36bbf3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:04.520388 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c4f1e-4d37-4446-837d-7d4ac48e1993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:04.521016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c4f1e-4d37-4446-837d-7d4ac48e1993 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:04.521874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c4f1e-4d37-4446-837d-7d4ac48e1993 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:04.522441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c4f1e-4d37-4446-837d-7d4ac48e1993 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:04.523491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c4f1e-4d37-4446-837d-7d4ac48e1993 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:04.523812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c4f1e-4d37-4446-837d-7d4ac48e1993 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:04.525490 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c4f1e-4d37-4446-837d-7d4ac48e1993 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:04.531961 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf262fe5-896c-42b7-abb8-03a24a19b782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:04.532769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf262fe5-896c-42b7-abb8-03a24a19b782 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:04.533245 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf262fe5-896c-42b7-abb8-03a24a19b782 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:04.533654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf262fe5-896c-42b7-abb8-03a24a19b782 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:04.534545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf262fe5-896c-42b7-abb8-03a24a19b782 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:04.535029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf262fe5-896c-42b7-abb8-03a24a19b782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:04.536583 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf262fe5-896c-42b7-abb8-03a24a19b782 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:04.539139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 354/703] 10.222.0.100 () {56 vars in 1129 bytes} [Mon Aug 1 15:01:04 2022] GET /identity/v3/services?type=block-storage => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:04.540727 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c4f1e-4d37-4446-837d-7d4ac48e1993 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:04.554049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 350/704] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:04.559975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf262fe5-896c-42b7-abb8-03a24a19b782 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:04.590986 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:04.681281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:04.688523 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c4f1e-4d37-4446-837d-7d4ac48e1993 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:04.776032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f855679-797f-4d5f-8fe6-ff573d7b8a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:04.776676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f855679-797f-4d5f-8fe6-ff573d7b8a01 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:04.776960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f855679-797f-4d5f-8fe6-ff573d7b8a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:04.777233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f855679-797f-4d5f-8fe6-ff573d7b8a01 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:04.778007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f855679-797f-4d5f-8fe6-ff573d7b8a01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:04.778321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f855679-797f-4d5f-8fe6-ff573d7b8a01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:04.779889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f855679-797f-4d5f-8fe6-ff573d7b8a01 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:04.781901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf262fe5-896c-42b7-abb8-03a24a19b782 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:04.793837 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 355/705] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:04.800391 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f855679-797f-4d5f-8fe6-ff573d7b8a01 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:04.825441 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac192a8-40f6-4451-9381-4ca81fa43def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:04.826099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac192a8-40f6-4451-9381-4ca81fa43def None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:04.826377 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac192a8-40f6-4451-9381-4ca81fa43def None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:04.826673 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac192a8-40f6-4451-9381-4ca81fa43def None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:04.833827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac192a8-40f6-4451-9381-4ca81fa43def None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:04.835358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac192a8-40f6-4451-9381-4ca81fa43def None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-aac192a8-40f6-4451-9381-4ca81fa43def None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:04.847794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:04.848786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:04.849784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:04.849784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:04.849784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:04.849784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:04.849784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:04.849784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:04.849784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:04.849784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:04.853760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 351/706] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:04.855104 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac192a8-40f6-4451-9381-4ca81fa43def None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:04.924136 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f855679-797f-4d5f-8fe6-ff573d7b8a01 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:04.964123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:04.988570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f7a46-d905-4df8-b174-ee2ee4980317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:04.989337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f7a46-d905-4df8-b174-ee2ee4980317 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:04.989670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f7a46-d905-4df8-b174-ee2ee4980317 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:04.990519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f7a46-d905-4df8-b174-ee2ee4980317 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:04.991578 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f7a46-d905-4df8-b174-ee2ee4980317 None admin] RBAC: Adding query filter params (role.id=70e0c42f1bdd49178db398637f021bbc, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=41a399a3cc7446c6a8dad94075851797) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:04.991895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f7a46-d905-4df8-b174-ee2ee4980317 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:04.993754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f7a46-d905-4df8-b174-ee2ee4980317 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:05.007102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac192a8-40f6-4451-9381-4ca81fa43def None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:05.015069 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 356/707] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:04 2022] GET /identity/v3/role_assignments?user.id=41a399a3cc7446c6a8dad94075851797&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=70e0c42f1bdd49178db398637f021bbc => generated 610 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:05.047197 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa020a5-97e0-422f-96f8-58fb7a4b03d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:05.047758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa020a5-97e0-422f-96f8-58fb7a4b03d7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:05.048043 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa020a5-97e0-422f-96f8-58fb7a4b03d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:05.048324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa020a5-97e0-422f-96f8-58fb7a4b03d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:05.049101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa020a5-97e0-422f-96f8-58fb7a4b03d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:05.049413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa020a5-97e0-422f-96f8-58fb7a4b03d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:05.050934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa020a5-97e0-422f-96f8-58fb7a4b03d7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:05.070852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 352/708] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:05.072406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f7a46-d905-4df8-b174-ee2ee4980317 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:05.108185 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:05.135134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f98378-f9a1-4446-93a0-172193b54fd0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:05.135465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f98378-f9a1-4446-93a0-172193b54fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:05.135704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f98378-f9a1-4446-93a0-172193b54fd0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:05.137530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 353/709] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:01:05.152329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99513c-8b60-43d1-a178-52c44ad28dbf None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:05.152675 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99513c-8b60-43d1-a178-52c44ad28dbf None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:05.152883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99513c-8b60-43d1-a178-52c44ad28dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:05.179609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-aa99513c-8b60-43d1-a178-52c44ad28dbf None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:05.180844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-aa99513c-8b60-43d1-a178-52c44ad28dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:05.247606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f7a46-d905-4df8-b174-ee2ee4980317 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:05.283668 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 354/710] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:01:05.285067 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23c0f7-681b-4c9c-a691-6cb52fe70170 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:05.285359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23c0f7-681b-4c9c-a691-6cb52fe70170 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:05.285635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23c0f7-681b-4c9c-a691-6cb52fe70170 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:05.287383 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 355/711] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:05.294174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcf45a-fc71-4ee3-a420-c812af126c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:05.294739 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209845c9-b359-47bd-afe0-cee7ee2b5536 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:05.294739 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209845c9-b359-47bd-afe0-cee7ee2b5536 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:05.294909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209845c9-b359-47bd-afe0-cee7ee2b5536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:05.295784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcf45a-fc71-4ee3-a420-c812af126c68 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:05.296247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcf45a-fc71-4ee3-a420-c812af126c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:05.296698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcf45a-fc71-4ee3-a420-c812af126c68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:05.297757 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcf45a-fc71-4ee3-a420-c812af126c68 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=ec50ac7c54324884a1b2a6c63d617907) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:05.298271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcf45a-fc71-4ee3-a420-c812af126c68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:05.299973 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcf45a-fc71-4ee3-a420-c812af126c68 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:05.313886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 357/712] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:05 2022] GET /identity/v3/role_assignments?user.id=ec50ac7c54324884a1b2a6c63d617907&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 260 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:05.315212 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcf45a-fc71-4ee3-a420-c812af126c68 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:05.332802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-209845c9-b359-47bd-afe0-cee7ee2b5536 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:05.335021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-209845c9-b359-47bd-afe0-cee7ee2b5536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:05.426421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:05.430127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:05.450726 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 356/713] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 01 15:01:05.453845 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcf45a-fc71-4ee3-a420-c812af126c68 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:05.471698 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209845c9-b359-47bd-afe0-cee7ee2b5536 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:05.494126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675beb96-30cb-46de-9aa9-6e11387b24d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:05.494803 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675beb96-30cb-46de-9aa9-6e11387b24d8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:05.495083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675beb96-30cb-46de-9aa9-6e11387b24d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:05.495355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675beb96-30cb-46de-9aa9-6e11387b24d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:05.508016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675beb96-30cb-46de-9aa9-6e11387b24d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:05.509488 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675beb96-30cb-46de-9aa9-6e11387b24d8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-675beb96-30cb-46de-9aa9-6e11387b24d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:05.525842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:05.526833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:05.527841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:05.527841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:05.527841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:05.527841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:05.531924 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 358/714] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:05.533270 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675beb96-30cb-46de-9aa9-6e11387b24d8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:05.649732 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675beb96-30cb-46de-9aa9-6e11387b24d8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:05.677018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209845c9-b359-47bd-afe0-cee7ee2b5536 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:05.692541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a4bbb-e5c5-4f13-9e4c-304ec6d55863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:05.694093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a4bbb-e5c5-4f13-9e4c-304ec6d55863 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:05.694647 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a4bbb-e5c5-4f13-9e4c-304ec6d55863 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:05.695800 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a4bbb-e5c5-4f13-9e4c-304ec6d55863 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:05.696700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a4bbb-e5c5-4f13-9e4c-304ec6d55863 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:05.699080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a4bbb-e5c5-4f13-9e4c-304ec6d55863 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:05.701397 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a4bbb-e5c5-4f13-9e4c-304ec6d55863 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:05.714114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:05.729825 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 359/715] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:05.734436 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a4bbb-e5c5-4f13-9e4c-304ec6d55863 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:05.764186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cddc8-6beb-407e-91c1-40316ef68795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:05.764854 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cddc8-6beb-407e-91c1-40316ef68795 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:05.765134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cddc8-6beb-407e-91c1-40316ef68795 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:05.765410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46cddc8-6beb-407e-91c1-40316ef68795 None admin] PATH_INFO: `/v3/services/network` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:05.766054 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cddc8-6beb-407e-91c1-40316ef68795 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:05.767480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46cddc8-6beb-407e-91c1-40316ef68795 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b46cddc8-6beb-407e-91c1-40316ef68795 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:05.795699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:05.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:05.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 357/716] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:05 2022] GET /identity/v3/services/network => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:05.803437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cddc8-6beb-407e-91c1-40316ef68795 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:05.903120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:05.912746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a4bbb-e5c5-4f13-9e4c-304ec6d55863 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:05.961890 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46cddc8-6beb-407e-91c1-40316ef68795 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:06.001300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933a93c-d6e4-4967-9bbf-9f4c9e3281e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:06.002719 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933a93c-d6e4-4967-9bbf-9f4c9e3281e3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:06.003175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933a93c-d6e4-4967-9bbf-9f4c9e3281e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:06.003685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f933a93c-d6e4-4967-9bbf-9f4c9e3281e3 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:06.004582 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933a93c-d6e4-4967-9bbf-9f4c9e3281e3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:06.005067 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933a93c-d6e4-4967-9bbf-9f4c9e3281e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:06.006865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f933a93c-d6e4-4967-9bbf-9f4c9e3281e3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:06.012874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 358/717] 10.222.0.100 () {56 vars in 1117 bytes} [Mon Aug 1 15:01:05 2022] GET /identity/v3/services?name=network => generated 125 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:06.017891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933a93c-d6e4-4967-9bbf-9f4c9e3281e3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:06.019048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b80e73-ab26-4468-8ce5-266bdcb5fe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:06.020241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b80e73-ab26-4468-8ce5-266bdcb5fe09 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:06.022592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b80e73-ab26-4468-8ce5-266bdcb5fe09 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:06.023176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b80e73-ab26-4468-8ce5-266bdcb5fe09 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:06.025036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b80e73-ab26-4468-8ce5-266bdcb5fe09 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:06.032336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b80e73-ab26-4468-8ce5-266bdcb5fe09 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:06.078386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 360/718] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:01:05 2022] POST /identity/v3/users => generated 255 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:01:06.146608 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f933a93c-d6e4-4967-9bbf-9f4c9e3281e3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:06.166285 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:06.186203 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d05d5-f9b9-4d53-89c3-d11de19da038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:06.187226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d05d5-f9b9-4d53-89c3-d11de19da038 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:06.188003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d05d5-f9b9-4d53-89c3-d11de19da038 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:06.188549 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d05d5-f9b9-4d53-89c3-d11de19da038 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:06.189453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d05d5-f9b9-4d53-89c3-d11de19da038 None admin] RBAC: Adding query filter params (type=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:06.189961 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d05d5-f9b9-4d53-89c3-d11de19da038 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:06.191619 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d05d5-f9b9-4d53-89c3-d11de19da038 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:06.198681 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 359/719] 10.222.0.100 () {56 vars in 1117 bytes} [Mon Aug 1 15:01:06 2022] GET /identity/v3/services?type=network => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:07.482966 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c687e31-eab4-4d77-86b0-8a4d6a703812 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:07.484001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c687e31-eab4-4d77-86b0-8a4d6a703812 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:07.484539 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c687e31-eab4-4d77-86b0-8a4d6a703812 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:07.486362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 361/720] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:07 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:07.492286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b47a20-5342-491c-b62f-7a67be7a7ae5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:07.492794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b47a20-5342-491c-b62f-7a67be7a7ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:07.492874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b47a20-5342-491c-b62f-7a67be7a7ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:07.520242 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-77b47a20-5342-491c-b62f-7a67be7a7ae5 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:07.521480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-77b47a20-5342-491c-b62f-7a67be7a7ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:07.625395 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 360/721] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:01:07.659228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c687e31-eab4-4d77-86b0-8a4d6a703812 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:07.745653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:07.820502 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c687e31-eab4-4d77-86b0-8a4d6a703812 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:07.821966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395750c2-ce8d-4e63-a53e-637788117c01 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:07.822461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395750c2-ce8d-4e63-a53e-637788117c01 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:07.822998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395750c2-ce8d-4e63-a53e-637788117c01 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:07.824806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 361/722] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:07 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:07.829893 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20080d5-134b-4327-9a81-4b7bff3f3412 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:07.830465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20080d5-134b-4327-9a81-4b7bff3f3412 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:07.830976 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20080d5-134b-4327-9a81-4b7bff3f3412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:07.864199 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f20080d5-134b-4327-9a81-4b7bff3f3412 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:07.866312 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f20080d5-134b-4327-9a81-4b7bff3f3412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:07.872474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d631ff4-5421-4335-8b67-23226d0b3b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:07.873335 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d631ff4-5421-4335-8b67-23226d0b3b01 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:07.873801 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d631ff4-5421-4335-8b67-23226d0b3b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:07.874284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d631ff4-5421-4335-8b67-23226d0b3b01 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:07.875189 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d631ff4-5421-4335-8b67-23226d0b3b01 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:07.876764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d631ff4-5421-4335-8b67-23226d0b3b01 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:07.908615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 362/723] 10.222.0.100 () {60 vars in 1146 bytes} [Mon Aug 1 15:01:07 2022] POST /identity/v3/services => generated 252 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:01:07.919502 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce611678-5520-4586-ab03-89f073b632e0 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:07.920203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce611678-5520-4586-ab03-89f073b632e0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:07.920737 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce611678-5520-4586-ab03-89f073b632e0 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:07.923060 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 363/724] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:07 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:01:07.929322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ed8ad-6f68-4eb9-81c1-04edb6994373 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:07.929906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ed8ad-6f68-4eb9-81c1-04edb6994373 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:07.930393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ed8ad-6f68-4eb9-81c1-04edb6994373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:07.967333 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-030ed8ad-6f68-4eb9-81c1-04edb6994373 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:07.969324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-030ed8ad-6f68-4eb9-81c1-04edb6994373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:08.011022 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 362/725] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:07 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:01:08.047175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20080d5-134b-4327-9a81-4b7bff3f3412 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:08.090381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 364/726] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:07 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 15:01:08.108415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:08.114332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ed8ad-6f68-4eb9-81c1-04edb6994373 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:08.170050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:08.187037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20080d5-134b-4327-9a81-4b7bff3f3412 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:08.229683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa1918-b53e-4909-a989-241972426e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:08.230347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa1918-b53e-4909-a989-241972426e6f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:08.230626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa1918-b53e-4909-a989-241972426e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:08.230955 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa1918-b53e-4909-a989-241972426e6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:08.232317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ed8ad-6f68-4eb9-81c1-04edb6994373 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:08.254817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa1918-b53e-4909-a989-241972426e6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-47fa1918-b53e-4909-a989-241972426e6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:08.261107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:08.262029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:08.262916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:08.262916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:08.262916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:08.262916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:08.262916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:08.262916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:08.262916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:08.262916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:08.262916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:08.262916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:08.262916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:08.262916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:08.262916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 363/727] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:01:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:08.269748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa1918-b53e-4909-a989-241972426e6f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:08.273222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd18a5a-f0b4-4533-a4fd-41d52975804b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:08.273874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd18a5a-f0b4-4533-a4fd-41d52975804b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:08.274164 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd18a5a-f0b4-4533-a4fd-41d52975804b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:08.274645 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd18a5a-f0b4-4533-a4fd-41d52975804b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:08.286276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd18a5a-f0b4-4533-a4fd-41d52975804b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-cdd18a5a-f0b4-4533-a4fd-41d52975804b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:08.291835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:08.294102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:08.295758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:08.295758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:08.295758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:08.295758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:08.295758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:08.295758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:08.295758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:08.295758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:08.295758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:08.295758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:08.295758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:08.295758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:08.295758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 365/728] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:01:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:08.303554 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd18a5a-f0b4-4533-a4fd-41d52975804b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:08.424929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:08.442092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa1918-b53e-4909-a989-241972426e6f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:08.448691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd18a5a-f0b4-4533-a4fd-41d52975804b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:08.454679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:08.512640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea368b-9711-4e22-b53c-11983af44207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:08.513591 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea368b-9711-4e22-b53c-11983af44207 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:08.513999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea368b-9711-4e22-b53c-11983af44207 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:08.514402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea368b-9711-4e22-b53c-11983af44207 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:08.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea368b-9711-4e22-b53c-11983af44207 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:08.515934 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea368b-9711-4e22-b53c-11983af44207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:08.517968 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea368b-9711-4e22-b53c-11983af44207 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:08.532944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 366/729] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:01:08 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:08.542181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea368b-9711-4e22-b53c-11983af44207 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:08.560074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fbde8-532b-4f83-b8ce-d65c9db97be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:08.560794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fbde8-532b-4f83-b8ce-d65c9db97be4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:08.561011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fbde8-532b-4f83-b8ce-d65c9db97be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:08.561287 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fbde8-532b-4f83-b8ce-d65c9db97be4 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:08.562028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fbde8-532b-4f83-b8ce-d65c9db97be4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:08.562337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fbde8-532b-4f83-b8ce-d65c9db97be4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:08.563843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fbde8-532b-4f83-b8ce-d65c9db97be4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:08.590640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 364/730] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:01:08 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:08.611760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fbde8-532b-4f83-b8ce-d65c9db97be4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:08.732449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea368b-9711-4e22-b53c-11983af44207 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:08.789421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488f74c-f854-4d4e-a902-a6e89c1a3eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:08.791168 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488f74c-f854-4d4e-a902-a6e89c1a3eff None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:08.791456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488f74c-f854-4d4e-a902-a6e89c1a3eff None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:08.791735 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488f74c-f854-4d4e-a902-a6e89c1a3eff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:08.815402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488f74c-f854-4d4e-a902-a6e89c1a3eff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:08.820217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488f74c-f854-4d4e-a902-a6e89c1a3eff None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:08.853827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fbde8-532b-4f83-b8ce-d65c9db97be4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-b488f74c-f854-4d4e-a902-a6e89c1a3eff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:08.854904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:08.855785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:08.856678 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:08.856678 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:08.856678 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:08.856678 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:08.859875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 367/731] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:08.861422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2910d-5e28-4e06-a2ea-27dc8e949505 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:08.861836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2910d-5e28-4e06-a2ea-27dc8e949505 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:08.862237 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2910d-5e28-4e06-a2ea-27dc8e949505 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:08.863993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 368/732] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:08.864920 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2910d-5e28-4e06-a2ea-27dc8e949505 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:08.901866 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:08.905604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e00d74-883a-4153-9c13-0f2af19a800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:08.906382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e00d74-883a-4153-9c13-0f2af19a800c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:08.906797 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e00d74-883a-4153-9c13-0f2af19a800c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:08.907265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e00d74-883a-4153-9c13-0f2af19a800c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:08.919959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e00d74-883a-4153-9c13-0f2af19a800c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:08.922805 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e00d74-883a-4153-9c13-0f2af19a800c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c8e00d74-883a-4153-9c13-0f2af19a800c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:08.944799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:08.946421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:08.948091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:08.948091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:08.948091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:08.948091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:08.948526 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 365/733] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:08.955722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12474fa7-9d50-4d57-9158-6fd91c88cb86 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:08.955722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12474fa7-9d50-4d57-9158-6fd91c88cb86 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:08.955722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12474fa7-9d50-4d57-9158-6fd91c88cb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:08.978194 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-12474fa7-9d50-4d57-9158-6fd91c88cb86 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:08.980188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-12474fa7-9d50-4d57-9158-6fd91c88cb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:08.990285 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:09.014461 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2910d-5e28-4e06-a2ea-27dc8e949505 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:09.054872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c2dc0-24d4-4774-8d86-ba89d1134686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:09.055741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c2dc0-24d4-4774-8d86-ba89d1134686 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:09.056162 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c2dc0-24d4-4774-8d86-ba89d1134686 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:09.056583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c2dc0-24d4-4774-8d86-ba89d1134686 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:09.057413 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c2dc0-24d4-4774-8d86-ba89d1134686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:09.057961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c2dc0-24d4-4774-8d86-ba89d1134686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:09.058455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 366/734] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:08 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:01:09.059345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12474fa7-9d50-4d57-9158-6fd91c88cb86 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:09.059955 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669c2dc0-24d4-4774-8d86-ba89d1134686 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:09.076764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 369/735] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:09.079128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501b5ce-66d7-4343-afa8-67552a4f212e None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:09.079561 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501b5ce-66d7-4343-afa8-67552a4f212e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:09.079999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501b5ce-66d7-4343-afa8-67552a4f212e None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:09.086344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 370/736] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:09 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:09.086344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501b5ce-66d7-4343-afa8-67552a4f212e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:09.171485 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12474fa7-9d50-4d57-9158-6fd91c88cb86 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:09.174522 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:09.198316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501b5ce-66d7-4343-afa8-67552a4f212e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:09.209833 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25730960-0484-43af-86ab-882de2f07404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:09.210822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25730960-0484-43af-86ab-882de2f07404 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:09.211273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25730960-0484-43af-86ab-882de2f07404 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:09.211563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25730960-0484-43af-86ab-882de2f07404 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:09.212296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25730960-0484-43af-86ab-882de2f07404 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:09.212864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25730960-0484-43af-86ab-882de2f07404 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:09.214249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25730960-0484-43af-86ab-882de2f07404 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:09.226267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 367/737] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:09.227074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25730960-0484-43af-86ab-882de2f07404 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:09.233229 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:09.240168 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd7a9f-5958-46e9-9527-33b5cc9adf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:09.240813 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd7a9f-5958-46e9-9527-33b5cc9adf9f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:09.241101 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd7a9f-5958-46e9-9527-33b5cc9adf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:09.241372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd7a9f-5958-46e9-9527-33b5cc9adf9f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:09.249066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd7a9f-5958-46e9-9527-33b5cc9adf9f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:09.250508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd7a9f-5958-46e9-9527-33b5cc9adf9f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-25cd7a9f-5958-46e9-9527-33b5cc9adf9f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:09.262326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:09.263496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:09.264610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:09.264610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:09.264610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:09.264610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:09.264610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:09.264610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:09.264610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:09.264610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:09.264610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:01:09.264610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:09.264610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 371/738] 10.222.0.100 () {56 vars in 1098 bytes} [Mon Aug 1 15:01:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:09.265200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ce6aa-7716-4d76-b217-e5558d8816f4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:09.265200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ce6aa-7716-4d76-b217-e5558d8816f4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:09.265333 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ce6aa-7716-4d76-b217-e5558d8816f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:09.283820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b74ce6aa-7716-4d76-b217-e5558d8816f4 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:09.285032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b74ce6aa-7716-4d76-b217-e5558d8816f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:09.378692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 372/739] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:09 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 15:01:09.378692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ce6aa-7716-4d76-b217-e5558d8816f4 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:09.470252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:09.501592 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25730960-0484-43af-86ab-882de2f07404 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:09.514790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ce6aa-7716-4d76-b217-e5558d8816f4 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:09.553051 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647b54e-c605-424e-a86a-9499752770df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:09.553951 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647b54e-c605-424e-a86a-9499752770df None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:09.554239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647b54e-c605-424e-a86a-9499752770df None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:09.554800 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647b54e-c605-424e-a86a-9499752770df None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:09.555953 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647b54e-c605-424e-a86a-9499752770df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:09.557822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647b54e-c605-424e-a86a-9499752770df None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:09.560143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebac242-16db-4d4a-8abb-f4e9ca857326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:09.560787 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebac242-16db-4d4a-8abb-f4e9ca857326 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:09.561066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebac242-16db-4d4a-8abb-f4e9ca857326 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:09.561338 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebac242-16db-4d4a-8abb-f4e9ca857326 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:09.570312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebac242-16db-4d4a-8abb-f4e9ca857326 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:09.572419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebac242-16db-4d4a-8abb-f4e9ca857326 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:09.582717 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 368/740] 10.222.0.100 () {60 vars in 1146 bytes} [Mon Aug 1 15:01:09 2022] POST /identity/v3/services => generated 241 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:01:09.582717 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647b54e-c605-424e-a86a-9499752770df None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-9ebac242-16db-4d4a-8abb-f4e9ca857326 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:09.594375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:09.595665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:09.596808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:09.596808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:09.596808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:09.596808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:09.596808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:09.596808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:09.596808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:09.596808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:09.596808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:01:09.596808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:09.604358 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 373/741] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:01:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:09.605590 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebac242-16db-4d4a-8abb-f4e9ca857326 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:09.703474 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:09.731958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebac242-16db-4d4a-8abb-f4e9ca857326 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:09.739011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647b54e-c605-424e-a86a-9499752770df None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:09.764882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:09.776158 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7aaa83-19c4-47bb-a7f9-8042925f8ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:09.777058 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7aaa83-19c4-47bb-a7f9-8042925f8ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:09.777595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7aaa83-19c4-47bb-a7f9-8042925f8ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:09.778017 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7aaa83-19c4-47bb-a7f9-8042925f8ff2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:09.788719 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceed9f9-d662-4f42-9c9e-efcc2615b0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:09.789346 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceed9f9-d662-4f42-9c9e-efcc2615b0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:09.789627 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceed9f9-d662-4f42-9c9e-efcc2615b0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:09.789904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceed9f9-d662-4f42-9c9e-efcc2615b0d8 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:09.790751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceed9f9-d662-4f42-9c9e-efcc2615b0d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:09.791064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceed9f9-d662-4f42-9c9e-efcc2615b0d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:09.792777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7aaa83-19c4-47bb-a7f9-8042925f8ff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:09.792897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceed9f9-d662-4f42-9c9e-efcc2615b0d8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-be7aaa83-19c4-47bb-a7f9-8042925f8ff2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:09.800002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:09.801042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:09.801890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:09.801890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:09.801890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:09.801890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:09.801890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:09.801890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:09.801890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:09.801890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:09.801890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:09.801890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:09.801890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:01:09.801890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:09.809704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 374/742] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:01:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:09.810893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7aaa83-19c4-47bb-a7f9-8042925f8ff2 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:09.813230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 369/743] 10.222.0.100 () {56 vars in 1138 bytes} [Mon Aug 1 15:01:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:09.814360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceed9f9-d662-4f42-9c9e-efcc2615b0d8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:09.929940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7aaa83-19c4-47bb-a7f9-8042925f8ff2 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:09.936442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceed9f9-d662-4f42-9c9e-efcc2615b0d8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:09.959988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:09.969393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149882b4-951c-4437-a86d-b893cabb4876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:09.970014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149882b4-951c-4437-a86d-b893cabb4876 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:09.970291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149882b4-951c-4437-a86d-b893cabb4876 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:09.970559 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149882b4-951c-4437-a86d-b893cabb4876 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:09.971387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149882b4-951c-4437-a86d-b893cabb4876 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:09.971694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149882b4-951c-4437-a86d-b893cabb4876 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:09.973145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149882b4-951c-4437-a86d-b893cabb4876 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:09.986469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da451d-3c61-4983-9645-65646e5247fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:09.987527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da451d-3c61-4983-9645-65646e5247fe None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:09.988545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da451d-3c61-4983-9645-65646e5247fe None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:09.989033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da451d-3c61-4983-9645-65646e5247fe None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:09.990836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da451d-3c61-4983-9645-65646e5247fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:09.991354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da451d-3c61-4983-9645-65646e5247fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:09.992407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 375/744] 10.222.0.100 () {56 vars in 1140 bytes} [Mon Aug 1 15:01:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:09.993323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149882b4-951c-4437-a86d-b893cabb4876 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:09.993532 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da451d-3c61-4983-9645-65646e5247fe None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:10.004765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 370/745] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:09 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:10.006853 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da451d-3c61-4983-9645-65646e5247fe None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:10.064770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:10.128814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149882b4-951c-4437-a86d-b893cabb4876 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:10.145483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da451d-3c61-4983-9645-65646e5247fe None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:10.168386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f85bb7-008b-47ef-897b-3f469e6c8c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:10.169512 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f85bb7-008b-47ef-897b-3f469e6c8c46 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:10.170011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f85bb7-008b-47ef-897b-3f469e6c8c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:10.170508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f85bb7-008b-47ef-897b-3f469e6c8c46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:10.183706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f85bb7-008b-47ef-897b-3f469e6c8c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:10.185865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f85bb7-008b-47ef-897b-3f469e6c8c46 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:10.188125 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dd250-5233-411d-96a4-67964e335352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:10.188759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dd250-5233-411d-96a4-67964e335352 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:10.189040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dd250-5233-411d-96a4-67964e335352 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:10.189402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6dd250-5233-411d-96a4-67964e335352 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-a5f85bb7-008b-47ef-897b-3f469e6c8c46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:10.207406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:10.208403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:10.209307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:10.209307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:10.209307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:10.209307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:10.216465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 376/746] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:10.217936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f85bb7-008b-47ef-897b-3f469e6c8c46 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:10.239363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:10.263400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6dd250-5233-411d-96a4-67964e335352 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:10.264862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6dd250-5233-411d-96a4-67964e335352 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:10.331517 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:10.337115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f85bb7-008b-47ef-897b-3f469e6c8c46 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:10.369888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 371/747] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:10 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:01:10.370889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dd250-5233-411d-96a4-67964e335352 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:10.378584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaa9e6-0b64-41f1-b9f0-2c560f135d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:10.379618 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaa9e6-0b64-41f1-b9f0-2c560f135d2e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:10.380304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaa9e6-0b64-41f1-b9f0-2c560f135d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:10.380781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaa9e6-0b64-41f1-b9f0-2c560f135d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:10.396311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaa9e6-0b64-41f1-b9f0-2c560f135d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:10.398352 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaa9e6-0b64-41f1-b9f0-2c560f135d2e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-3ecaa9e6-0b64-41f1-b9f0-2c560f135d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:10.415221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:10.416202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:10.417115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:10.417115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:10.417115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:10.417115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:10.419717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 377/748] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:10.420829 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaa9e6-0b64-41f1-b9f0-2c560f135d2e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:10.457163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:10.481418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6dd250-5233-411d-96a4-67964e335352 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:10.517571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c435a66-f127-40fc-8e51-87ca47ba4bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:10.518221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c435a66-f127-40fc-8e51-87ca47ba4bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:10.518500 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c435a66-f127-40fc-8e51-87ca47ba4bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:10.518795 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c435a66-f127-40fc-8e51-87ca47ba4bc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:10.519486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c435a66-f127-40fc-8e51-87ca47ba4bc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:10.519794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c435a66-f127-40fc-8e51-87ca47ba4bc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:10.521537 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c435a66-f127-40fc-8e51-87ca47ba4bc0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:10.533656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 372/749] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:10.534499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c435a66-f127-40fc-8e51-87ca47ba4bc0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:10.554396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaa9e6-0b64-41f1-b9f0-2c560f135d2e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:10.592188 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5693c70d-8ceb-4157-a0a3-d877a9d7a6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:10.592871 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693c70d-8ceb-4157-a0a3-d877a9d7a6ad None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:10.593181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693c70d-8ceb-4157-a0a3-d877a9d7a6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:10.593478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5693c70d-8ceb-4157-a0a3-d877a9d7a6ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:10.606044 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693c70d-8ceb-4157-a0a3-d877a9d7a6ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:10.607554 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5693c70d-8ceb-4157-a0a3-d877a9d7a6ad None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-5693c70d-8ceb-4157-a0a3-d877a9d7a6ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:10.622902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:10.623893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:10.624787 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:10.624787 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:10.624787 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:10.624787 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:10.624787 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 378/750] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:10.625497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d49621-08de-490e-bc40-580f0c81d626 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:10.625601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d49621-08de-490e-bc40-580f0c81d626 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:10.625815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d49621-08de-490e-bc40-580f0c81d626 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:10.627247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 379/751] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:10 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:10.627802 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d49621-08de-490e-bc40-580f0c81d626 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:10.630614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:10.654226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c435a66-f127-40fc-8e51-87ca47ba4bc0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:10.697198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af0636-4ff5-42a9-a674-aeb2a14b55b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:10.697843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af0636-4ff5-42a9-a674-aeb2a14b55b0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:10.698128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af0636-4ff5-42a9-a674-aeb2a14b55b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:10.698402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af0636-4ff5-42a9-a674-aeb2a14b55b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:10.699163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af0636-4ff5-42a9-a674-aeb2a14b55b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:10.699476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af0636-4ff5-42a9-a674-aeb2a14b55b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:10.700949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af0636-4ff5-42a9-a674-aeb2a14b55b0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:10.709721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:10.723310 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 373/752] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:10.723310 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af0636-4ff5-42a9-a674-aeb2a14b55b0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:10.837383 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af0636-4ff5-42a9-a674-aeb2a14b55b0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:10.864175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d49621-08de-490e-bc40-580f0c81d626 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:10.877413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102010d-ef1f-454f-859f-4de8569b4332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:10.878040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102010d-ef1f-454f-859f-4de8569b4332 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:10.878322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102010d-ef1f-454f-859f-4de8569b4332 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:10.878593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102010d-ef1f-454f-859f-4de8569b4332 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:10.879329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102010d-ef1f-454f-859f-4de8569b4332 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:10.879748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102010d-ef1f-454f-859f-4de8569b4332 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:10.881888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102010d-ef1f-454f-859f-4de8569b4332 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:10.900426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 374/753] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:10.901945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c1d0e-8d5a-45fd-94a3-aa3f9e994293 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:10.902247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c1d0e-8d5a-45fd-94a3-aa3f9e994293 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:10.902526 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c1d0e-8d5a-45fd-94a3-aa3f9e994293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:10.904012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd7b3b-0123-44c4-be79-6fb2c0ab112c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:10.904676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd7b3b-0123-44c4-be79-6fb2c0ab112c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:10.904960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd7b3b-0123-44c4-be79-6fb2c0ab112c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:10.905238 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd7b3b-0123-44c4-be79-6fb2c0ab112c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:10.912984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd7b3b-0123-44c4-be79-6fb2c0ab112c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:10.914450 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffd7b3b-0123-44c4-be79-6fb2c0ab112c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:10.921465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fe1c1d0e-8d5a-45fd-94a3-aa3f9e994293 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:10.922739 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fe1c1d0e-8d5a-45fd-94a3-aa3f9e994293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2ffd7b3b-0123-44c4-be79-6fb2c0ab112c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:10.926592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:10.928202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:10.929109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:10.929109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:10.929109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:10.929109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:10.929109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:10.929109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:10.929109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:10.929109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:10.929109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 380/754] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:10.929592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd7b3b-0123-44c4-be79-6fb2c0ab112c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:10.987078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:10.994655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:11.014845 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 375/755] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:10 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:11.015915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c1d0e-8d5a-45fd-94a3-aa3f9e994293 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:11.039304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffd7b3b-0123-44c4-be79-6fb2c0ab112c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:11.075550 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97241f-7e8e-44fa-a5c6-271f17207354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:11.076205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97241f-7e8e-44fa-a5c6-271f17207354 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:11.076491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97241f-7e8e-44fa-a5c6-271f17207354 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:11.076777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97241f-7e8e-44fa-a5c6-271f17207354 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:11.085211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97241f-7e8e-44fa-a5c6-271f17207354 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:11.086702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97241f-7e8e-44fa-a5c6-271f17207354 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-0d97241f-7e8e-44fa-a5c6-271f17207354 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:11.099679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:11.100676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:11.101669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:11.101669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:11.101669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:11.101669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:11.101669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:11.101669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:11.101669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:11.101669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:11.101669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:01:11.101669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:11.105561 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 381/756] 10.222.0.100 () {56 vars in 1122 bytes} [Mon Aug 1 15:01:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:11.106320 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97241f-7e8e-44fa-a5c6-271f17207354 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:11.134688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c1d0e-8d5a-45fd-94a3-aa3f9e994293 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:11.180513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becde441-2b52-45f6-b0ae-8d1fc2a95fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:11.181171 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becde441-2b52-45f6-b0ae-8d1fc2a95fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:11.181453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becde441-2b52-45f6-b0ae-8d1fc2a95fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:11.181730 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becde441-2b52-45f6-b0ae-8d1fc2a95fe5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:11.189896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becde441-2b52-45f6-b0ae-8d1fc2a95fe5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:11.191721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becde441-2b52-45f6-b0ae-8d1fc2a95fe5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-becde441-2b52-45f6-b0ae-8d1fc2a95fe5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:11.204210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:11.205139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:11.206094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:11.206094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:11.206094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:11.206094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:11.206094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:11.206094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:11.206094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:11.206094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:11.206094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 376/757] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:11.206094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becde441-2b52-45f6-b0ae-8d1fc2a95fe5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:11.217068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97241f-7e8e-44fa-a5c6-271f17207354 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:11.244797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:11.257560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037fdd8-34e8-4cd3-8b1c-b5f8a8832eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:11.258435 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037fdd8-34e8-4cd3-8b1c-b5f8a8832eed None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:11.259081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037fdd8-34e8-4cd3-8b1c-b5f8a8832eed None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:11.259081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037fdd8-34e8-4cd3-8b1c-b5f8a8832eed None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:11.259747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037fdd8-34e8-4cd3-8b1c-b5f8a8832eed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:11.260052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037fdd8-34e8-4cd3-8b1c-b5f8a8832eed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:11.261487 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037fdd8-34e8-4cd3-8b1c-b5f8a8832eed None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:11.269301 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:11.277586 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 382/758] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:11.279584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037fdd8-34e8-4cd3-8b1c-b5f8a8832eed None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:11.330200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becde441-2b52-45f6-b0ae-8d1fc2a95fe5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:11.367878 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534234e8-e1c5-4628-8593-b7e1f71e6065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:11.368451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534234e8-e1c5-4628-8593-b7e1f71e6065 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:11.368733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534234e8-e1c5-4628-8593-b7e1f71e6065 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:11.369008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534234e8-e1c5-4628-8593-b7e1f71e6065 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:11.369635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534234e8-e1c5-4628-8593-b7e1f71e6065 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:11.371075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534234e8-e1c5-4628-8593-b7e1f71e6065 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-534234e8-e1c5-4628-8593-b7e1f71e6065 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:11.380437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:11.381450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:01:11.382280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:11.386715 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 377/759] 10.222.0.100 () {56 vars in 1134 bytes} [Mon Aug 1 15:01:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:11.387656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534234e8-e1c5-4628-8593-b7e1f71e6065 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:11.402056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037fdd8-34e8-4cd3-8b1c-b5f8a8832eed None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:11.447115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451ad3f-5f4a-4317-a309-7fdc165d3cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:11.447916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451ad3f-5f4a-4317-a309-7fdc165d3cdb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:11.448030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451ad3f-5f4a-4317-a309-7fdc165d3cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:11.448271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451ad3f-5f4a-4317-a309-7fdc165d3cdb None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:11.449070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451ad3f-5f4a-4317-a309-7fdc165d3cdb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:11.449377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451ad3f-5f4a-4317-a309-7fdc165d3cdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:11.451443 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451ad3f-5f4a-4317-a309-7fdc165d3cdb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:11.473478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 383/760] 10.222.0.100 () {56 vars in 1154 bytes} [Mon Aug 1 15:01:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:11.475283 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451ad3f-5f4a-4317-a309-7fdc165d3cdb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:11.508081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534234e8-e1c5-4628-8593-b7e1f71e6065 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:11.520533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:11.524312 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:11.552838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a048f03-dae7-4bca-b437-7d939751b7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:11.553562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a048f03-dae7-4bca-b437-7d939751b7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:11.553852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a048f03-dae7-4bca-b437-7d939751b7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:11.554128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a048f03-dae7-4bca-b437-7d939751b7d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:11.554916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a048f03-dae7-4bca-b437-7d939751b7d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:11.555337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a048f03-dae7-4bca-b437-7d939751b7d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:11.556777 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a048f03-dae7-4bca-b437-7d939751b7d8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:11.568819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 378/761] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:11.569637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a048f03-dae7-4bca-b437-7d939751b7d8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:11.594017 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451ad3f-5f4a-4317-a309-7fdc165d3cdb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:11.631093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8287db2-8d22-47d5-9ce5-247a8506ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:11.631684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8287db2-8d22-47d5-9ce5-247a8506ec09 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:11.631968 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8287db2-8d22-47d5-9ce5-247a8506ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:11.632248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8287db2-8d22-47d5-9ce5-247a8506ec09 None admin] PATH_INFO: `/v3/projects/d6e147b6607b481591ceb250f7208c16/users/ec50ac7c54324884a1b2a6c63d617907/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:11.634226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8287db2-8d22-47d5-9ce5-247a8506ec09 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6e147b6607b481591ceb250f7208c16, user_id=ec50ac7c54324884a1b2a6c63d617907, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:11.635699 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8287db2-8d22-47d5-9ce5-247a8506ec09 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:11.650964 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 384/762] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:01:11 2022] PUT /identity/v3/projects/d6e147b6607b481591ceb250f7208c16/users/ec50ac7c54324884a1b2a6c63d617907/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:01:11.651809 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8287db2-8d22-47d5-9ce5-247a8506ec09 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:11.701362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a048f03-dae7-4bca-b437-7d939751b7d8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:11.758388 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d747df-9fa7-498f-a133-6c807fdac89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:11.759081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d747df-9fa7-498f-a133-6c807fdac89f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:11.759396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d747df-9fa7-498f-a133-6c807fdac89f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:11.760276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d747df-9fa7-498f-a133-6c807fdac89f None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:11.761796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d747df-9fa7-498f-a133-6c807fdac89f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:11.762132 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d747df-9fa7-498f-a133-6c807fdac89f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:11.763580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d747df-9fa7-498f-a133-6c807fdac89f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:11.774458 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 379/763] 10.222.0.100 () {56 vars in 1129 bytes} [Mon Aug 1 15:01:11 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:11.775461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d747df-9fa7-498f-a133-6c807fdac89f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:11.824276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:11.828543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:11.829713 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8287db2-8d22-47d5-9ce5-247a8506ec09 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:11.882839 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f634291-8676-4f00-8c9b-2ea2b92a89e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:11.883480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f634291-8676-4f00-8c9b-2ea2b92a89e5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:11.883828 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f634291-8676-4f00-8c9b-2ea2b92a89e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:11.884173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f634291-8676-4f00-8c9b-2ea2b92a89e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:11.910585 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d747df-9fa7-498f-a133-6c807fdac89f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:11.953048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62bcc1-28cd-4095-b64d-e8da2435bc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:11.953714 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62bcc1-28cd-4095-b64d-e8da2435bc17 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:11.954041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62bcc1-28cd-4095-b64d-e8da2435bc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:11.954319 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62bcc1-28cd-4095-b64d-e8da2435bc17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:11.955043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f634291-8676-4f00-8c9b-2ea2b92a89e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:11.955340 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62bcc1-28cd-4095-b64d-e8da2435bc17 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=41a399a3cc7446c6a8dad94075851797) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:11.955645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62bcc1-28cd-4095-b64d-e8da2435bc17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:11.956532 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f634291-8676-4f00-8c9b-2ea2b92a89e5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:11.957099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62bcc1-28cd-4095-b64d-e8da2435bc17 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:11.966717 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 380/764] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:11 2022] GET /identity/v3/role_assignments?user.id=41a399a3cc7446c6a8dad94075851797&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:11.966717 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62bcc1-28cd-4095-b64d-e8da2435bc17 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:12.103843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:12.106118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:12.109684 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62bcc1-28cd-4095-b64d-e8da2435bc17 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:12.118615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 385/765] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:11 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:01:12.120469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b7c78-27cf-47c1-b6b3-1fe6d6821d0a None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:12.120989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b7c78-27cf-47c1-b6b3-1fe6d6821d0a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:12.121528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b7c78-27cf-47c1-b6b3-1fe6d6821d0a None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:12.124177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 386/766] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:12 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:12.125491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b7c78-27cf-47c1-b6b3-1fe6d6821d0a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:12.167520 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6f1c04-0ddb-4909-8ba0-41741051395d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:12.168389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f1c04-0ddb-4909-8ba0-41741051395d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:12.169261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f1c04-0ddb-4909-8ba0-41741051395d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:12.169550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6f1c04-0ddb-4909-8ba0-41741051395d None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:12.170420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f1c04-0ddb-4909-8ba0-41741051395d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:12.170792 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f1c04-0ddb-4909-8ba0-41741051395d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:12.172285 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6f1c04-0ddb-4909-8ba0-41741051395d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:12.179332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 381/767] 10.222.0.100 () {56 vars in 1129 bytes} [Mon Aug 1 15:01:11 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:12.180841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827b6e4-2849-48b1-a010-c0db7f62e928 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:12.181208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827b6e4-2849-48b1-a010-c0db7f62e928 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:12.181495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827b6e4-2849-48b1-a010-c0db7f62e928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:12.202644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2827b6e4-2849-48b1-a010-c0db7f62e928 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:12.204053 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2827b6e4-2849-48b1-a010-c0db7f62e928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:12.264458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b7c78-27cf-47c1-b6b3-1fe6d6821d0a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:12.295196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 382/768] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:12 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:12.296285 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827b6e4-2849-48b1-a010-c0db7f62e928 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:12.302415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129533c-8d91-48b3-991d-ac2426f48e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:12.303158 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129533c-8d91-48b3-991d-ac2426f48e44 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:12.303445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129533c-8d91-48b3-991d-ac2426f48e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:12.303780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129533c-8d91-48b3-991d-ac2426f48e44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:12.315775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129533c-8d91-48b3-991d-ac2426f48e44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:12.317259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5129533c-8d91-48b3-991d-ac2426f48e44 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-5129533c-8d91-48b3-991d-ac2426f48e44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:12.334426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:12.335504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:12.336408 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:12.336408 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:12.336408 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:12.336408 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:12.336408 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 387/769] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:12.339975 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129533c-8d91-48b3-991d-ac2426f48e44 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:12.401000 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:12.426760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:12.435510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2827b6e4-2849-48b1-a010-c0db7f62e928 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:12.453713 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129533c-8d91-48b3-991d-ac2426f48e44 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:12.481207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9dd4b-4974-47df-9a9f-5de32427c034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:12.481849 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9dd4b-4974-47df-9a9f-5de32427c034 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:12.482131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9dd4b-4974-47df-9a9f-5de32427c034 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:12.482404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9dd4b-4974-47df-9a9f-5de32427c034 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:12.483226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9dd4b-4974-47df-9a9f-5de32427c034 None admin] RBAC: Adding query filter params (interface=public, service_id=c854c71948514c68b4c0af86a7b0aa58, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:12.483642 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9dd4b-4974-47df-9a9f-5de32427c034 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:12.485154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9dd4b-4974-47df-9a9f-5de32427c034 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:12.494253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 383/770] 10.222.0.100 () {56 vars in 1256 bytes} [Mon Aug 1 15:01:12 2022] GET /identity/v3/endpoints?service_id=c854c71948514c68b4c0af86a7b0aa58&interface=public®ion_id=RegionOne => generated 195 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:12.495158 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9dd4b-4974-47df-9a9f-5de32427c034 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:12.498026 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05749133-f0a5-4a17-ac8c-35d473aebc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:12.499220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05749133-f0a5-4a17-ac8c-35d473aebc9c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:12.499731 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05749133-f0a5-4a17-ac8c-35d473aebc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:12.500262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05749133-f0a5-4a17-ac8c-35d473aebc9c None admin] PATH_INFO: `/v3/services/network` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:12.501147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05749133-f0a5-4a17-ac8c-35d473aebc9c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:12.502803 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05749133-f0a5-4a17-ac8c-35d473aebc9c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-05749133-f0a5-4a17-ac8c-35d473aebc9c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:12.513528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:12.514595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:12.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:12.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:12.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:12.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:12.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:12.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:12.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:12.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:12.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:12.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:12.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:12.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:12.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:01:12.515498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:12.518465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 388/771] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:12 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:12.519633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05749133-f0a5-4a17-ac8c-35d473aebc9c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:12.617975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9dd4b-4974-47df-9a9f-5de32427c034 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:12.645323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05749133-f0a5-4a17-ac8c-35d473aebc9c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:12.656707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445b053-514b-4302-b668-445f7f2eb6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:12.657340 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445b053-514b-4302-b668-445f7f2eb6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:12.657726 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445b053-514b-4302-b668-445f7f2eb6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:12.658008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445b053-514b-4302-b668-445f7f2eb6f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:12.658749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445b053-514b-4302-b668-445f7f2eb6f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:12.659062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445b053-514b-4302-b668-445f7f2eb6f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:12.660579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445b053-514b-4302-b668-445f7f2eb6f0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:12.664122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:12.675869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 384/772] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:12.676748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445b053-514b-4302-b668-445f7f2eb6f0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:12.690410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81ce1f-bd9f-481c-b0a9-df446bc9f41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:12.691080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81ce1f-bd9f-481c-b0a9-df446bc9f41e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:12.691428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81ce1f-bd9f-481c-b0a9-df446bc9f41e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:12.691769 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81ce1f-bd9f-481c-b0a9-df446bc9f41e None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:12.692498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81ce1f-bd9f-481c-b0a9-df446bc9f41e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:12.694703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81ce1f-bd9f-481c-b0a9-df446bc9f41e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:12.698131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:12.711002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 389/773] 10.222.0.100 () {56 vars in 1092 bytes} [Mon Aug 1 15:01:12 2022] GET /identity/v3/services => generated 776 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:12.712857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81ce1f-bd9f-481c-b0a9-df446bc9f41e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:12.794070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445b053-514b-4302-b668-445f7f2eb6f0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:12.841713 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0905eee-782c-479a-b653-357fa7b60882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:12.842352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0905eee-782c-479a-b653-357fa7b60882 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:12.842641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0905eee-782c-479a-b653-357fa7b60882 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:12.842940 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0905eee-782c-479a-b653-357fa7b60882 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:12.843635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0905eee-782c-479a-b653-357fa7b60882 None admin] RBAC: Adding query filter params (name=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:12.843947 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0905eee-782c-479a-b653-357fa7b60882 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:12.845454 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0905eee-782c-479a-b653-357fa7b60882 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:12.856131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 385/774] 10.222.0.100 () {56 vars in 1117 bytes} [Mon Aug 1 15:01:12 2022] GET /identity/v3/services?name=network => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:12.861899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0905eee-782c-479a-b653-357fa7b60882 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:12.878629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81ce1f-bd9f-481c-b0a9-df446bc9f41e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:12.918439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2a77e-ed6e-4580-a99d-009833b08376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:12.919378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2a77e-ed6e-4580-a99d-009833b08376 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:12.920180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2a77e-ed6e-4580-a99d-009833b08376 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:12.920599 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2a77e-ed6e-4580-a99d-009833b08376 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:12.928659 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2a77e-ed6e-4580-a99d-009833b08376 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:12.930420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2a77e-ed6e-4580-a99d-009833b08376 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-16f2a77e-ed6e-4580-a99d-009833b08376 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:12.945593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:12.946778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:12.947709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:12.947709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:12.947709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:12.947709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:12.947709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:12.947709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:12.947709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:12.947709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:12.954432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 390/775] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:12.956213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2a77e-ed6e-4580-a99d-009833b08376 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:12.999123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:13.019034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:13.036964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0905eee-782c-479a-b653-357fa7b60882 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:13.098253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481d270-593c-4ed0-9bd3-cd3bd3b03cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:13.099136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481d270-593c-4ed0-9bd3-cd3bd3b03cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:13.099523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481d270-593c-4ed0-9bd3-cd3bd3b03cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:13.099959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481d270-593c-4ed0-9bd3-cd3bd3b03cd3 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:13.101321 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481d270-593c-4ed0-9bd3-cd3bd3b03cd3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:13.101321 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481d270-593c-4ed0-9bd3-cd3bd3b03cd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:13.102729 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4481d270-593c-4ed0-9bd3-cd3bd3b03cd3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:13.111240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 386/776] 10.222.0.100 () {56 vars in 1117 bytes} [Mon Aug 1 15:01:12 2022] GET /identity/v3/services?type=network => generated 352 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:13.111465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2a77e-ed6e-4580-a99d-009833b08376 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:13.117912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481d270-593c-4ed0-9bd3-cd3bd3b03cd3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:13.162404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046afa49-555e-49a0-8ab7-14f2b761e576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:13.163370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046afa49-555e-49a0-8ab7-14f2b761e576 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:13.163932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046afa49-555e-49a0-8ab7-14f2b761e576 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:13.164529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046afa49-555e-49a0-8ab7-14f2b761e576 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:13.165958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046afa49-555e-49a0-8ab7-14f2b761e576 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:13.166605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046afa49-555e-49a0-8ab7-14f2b761e576 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:13.169542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046afa49-555e-49a0-8ab7-14f2b761e576 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:13.189248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 391/777] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:13.193515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046afa49-555e-49a0-8ab7-14f2b761e576 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:13.262485 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4481d270-593c-4ed0-9bd3-cd3bd3b03cd3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:13.309016 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639531e4-47ac-4ecd-8a22-8f659977f21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:13.309950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639531e4-47ac-4ecd-8a22-8f659977f21f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:13.310490 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639531e4-47ac-4ecd-8a22-8f659977f21f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:13.310945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639531e4-47ac-4ecd-8a22-8f659977f21f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:13.311922 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639531e4-47ac-4ecd-8a22-8f659977f21f None admin] RBAC: Adding query filter params (interface=public, service_id=b1dfdf1a499c4e79bc479c138a58dffe, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:13.313526 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639531e4-47ac-4ecd-8a22-8f659977f21f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:13.320678 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639531e4-47ac-4ecd-8a22-8f659977f21f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:13.336820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 387/778] 10.222.0.100 () {56 vars in 1256 bytes} [Mon Aug 1 15:01:13 2022] GET /identity/v3/endpoints?service_id=b1dfdf1a499c4e79bc479c138a58dffe&interface=public®ion_id=RegionOne => generated 195 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:13.342378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639531e4-47ac-4ecd-8a22-8f659977f21f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:13.371034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:13.397435 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:13.402037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046afa49-555e-49a0-8ab7-14f2b761e576 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:13.445409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d12e61d-3c98-4694-8fdc-f759b476789a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:13.446314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12e61d-3c98-4694-8fdc-f759b476789a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:13.446769 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12e61d-3c98-4694-8fdc-f759b476789a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:13.448183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d12e61d-3c98-4694-8fdc-f759b476789a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:13.449321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12e61d-3c98-4694-8fdc-f759b476789a None admin] RBAC: Adding query filter params (role.id=70e0c42f1bdd49178db398637f021bbc, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=d0723b88d7cd481cb962072f75d835c7) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:13.449876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12e61d-3c98-4694-8fdc-f759b476789a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:13.451609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d12e61d-3c98-4694-8fdc-f759b476789a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:13.463750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 392/779] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:13 2022] GET /identity/v3/role_assignments?user.id=d0723b88d7cd481cb962072f75d835c7&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=70e0c42f1bdd49178db398637f021bbc => generated 260 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:13.606800 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639531e4-47ac-4ecd-8a22-8f659977f21f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:13.668750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b7026-62bd-4ddf-84b1-5fc7938c4c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:13.669385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b7026-62bd-4ddf-84b1-5fc7938c4c3f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:13.669662 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b7026-62bd-4ddf-84b1-5fc7938c4c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:13.669933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b7026-62bd-4ddf-84b1-5fc7938c4c3f None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:13.670600 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b7026-62bd-4ddf-84b1-5fc7938c4c3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:13.672020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b7026-62bd-4ddf-84b1-5fc7938c4c3f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:13.684070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 388/780] 10.222.0.100 () {56 vars in 1092 bytes} [Mon Aug 1 15:01:13 2022] GET /identity/v3/services => generated 776 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:13.953963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9d120-e694-4b0c-84c5-86d524e12fe8 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:13.955067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9d120-e694-4b0c-84c5-86d524e12fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:13.955555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9d120-e694-4b0c-84c5-86d524e12fe8 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:13.957469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 393/781] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:13 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:13.963489 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2186c5f-ac71-4da5-b963-ac85e7b36fbd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:13.963980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2186c5f-ac71-4da5-b963-ac85e7b36fbd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:13.964077 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2186c5f-ac71-4da5-b963-ac85e7b36fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:13.986596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d2186c5f-ac71-4da5-b963-ac85e7b36fbd None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:13.987833 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d2186c5f-ac71-4da5-b963-ac85e7b36fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:14.117804 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 389/782] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:13 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:14.170766 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9d120-e694-4b0c-84c5-86d524e12fe8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:14.312387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9d120-e694-4b0c-84c5-86d524e12fe8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:14.328327 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bfce2-fab0-4754-ab1f-f47a5674b896 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:14.328327 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bfce2-fab0-4754-ab1f-f47a5674b896 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:14.328327 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bfce2-fab0-4754-ab1f-f47a5674b896 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:14.328906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 390/783] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:14.335082 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a0fdd-5001-4084-996e-ce3b4b118412 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:14.335257 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a0fdd-5001-4084-996e-ce3b4b118412 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:14.335454 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a0fdd-5001-4084-996e-ce3b4b118412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:14.345050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:14.365886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aca57-b429-42c0-8953-9885227258e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:14.367972 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aca57-b429-42c0-8953-9885227258e7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:14.368457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aca57-b429-42c0-8953-9885227258e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:14.369193 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02aca57-b429-42c0-8953-9885227258e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:14.373549 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-974a0fdd-5001-4084-996e-ce3b4b118412 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:14.376274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-974a0fdd-5001-4084-996e-ce3b4b118412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:14.395446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02aca57-b429-42c0-8953-9885227258e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-a02aca57-b429-42c0-8953-9885227258e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:14.402718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:14.404144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:14.404990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:14.404990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:14.404990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:14.404990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:14.404990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:14.404990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:14.404990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:14.404990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:14.404990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:14.404990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:14.404990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:14.404990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:14.404990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 394/784] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:01:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:14.409510 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aca57-b429-42c0-8953-9885227258e7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:14.446061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:14.530081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02aca57-b429-42c0-8953-9885227258e7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:14.573247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 391/785] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:14 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:14.586200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e968051-96cc-498e-b667-e58cb5da71c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:14.587098 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e968051-96cc-498e-b667-e58cb5da71c3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:14.587786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e968051-96cc-498e-b667-e58cb5da71c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:14.588202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e968051-96cc-498e-b667-e58cb5da71c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:14.589069 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e968051-96cc-498e-b667-e58cb5da71c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:14.589541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e968051-96cc-498e-b667-e58cb5da71c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:14.594372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e968051-96cc-498e-b667-e58cb5da71c3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:14.604765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a0fdd-5001-4084-996e-ce3b4b118412 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:14.605021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 395/786] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:01:14 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:14.612699 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e968051-96cc-498e-b667-e58cb5da71c3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:14.766973 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:14.791444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e968051-96cc-498e-b667-e58cb5da71c3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:14.808655 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a0fdd-5001-4084-996e-ce3b4b118412 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:14.852785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6fab9-1d4e-48cc-854f-b89b124e9423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:14.853817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6fab9-1d4e-48cc-854f-b89b124e9423 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:14.854318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6fab9-1d4e-48cc-854f-b89b124e9423 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:14.856208 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6fab9-1d4e-48cc-854f-b89b124e9423 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:14.879565 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344d870-3747-4674-a7e8-916e9fb8cf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:14.881269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6fab9-1d4e-48cc-854f-b89b124e9423 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:14.882129 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344d870-3747-4674-a7e8-916e9fb8cf0a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:14.883843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6fab9-1d4e-48cc-854f-b89b124e9423 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:14.884350 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344d870-3747-4674-a7e8-916e9fb8cf0a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:14.884821 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344d870-3747-4674-a7e8-916e9fb8cf0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:14.896463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:14.905495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344d870-3747-4674-a7e8-916e9fb8cf0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7344d870-3747-4674-a7e8-916e9fb8cf0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:14.910695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:14.911790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:14.912640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:14.912640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:14.912640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:14.912640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:14.912640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:14.912640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:14.912640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:14.912640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:14.912640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:14.912640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:14.912640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:14.912640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-91f6fab9-1d4e-48cc-854f-b89b124e9423 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:14.913797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:14.914959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:14.917232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:14.917232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:14.917232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:14.917232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:14.917232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 396/787] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:14.922748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 392/788] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:01:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:14.923068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6fab9-1d4e-48cc-854f-b89b124e9423 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:14.925616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344d870-3747-4674-a7e8-916e9fb8cf0a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:15.127835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6fab9-1d4e-48cc-854f-b89b124e9423 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:15.148855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344d870-3747-4674-a7e8-916e9fb8cf0a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:15.194914 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5c895-c34d-4bec-8f5a-f137b4c49d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:15.196491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5c895-c34d-4bec-8f5a-f137b4c49d7a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:15.197100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5c895-c34d-4bec-8f5a-f137b4c49d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:15.198636 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5c895-c34d-4bec-8f5a-f137b4c49d7a None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:15.199567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5c895-c34d-4bec-8f5a-f137b4c49d7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:15.200783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5c895-c34d-4bec-8f5a-f137b4c49d7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:15.202951 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5c895-c34d-4bec-8f5a-f137b4c49d7a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:15.210065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6ab50-58c9-47d6-8008-db1f1368b22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:15.210996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6ab50-58c9-47d6-8008-db1f1368b22d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:15.215063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6ab50-58c9-47d6-8008-db1f1368b22d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:15.215574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6ab50-58c9-47d6-8008-db1f1368b22d None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:15.216612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6ab50-58c9-47d6-8008-db1f1368b22d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:15.217160 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6ab50-58c9-47d6-8008-db1f1368b22d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:15.219379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6ab50-58c9-47d6-8008-db1f1368b22d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:15.226770 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 393/789] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:01:14 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:15.231342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5c895-c34d-4bec-8f5a-f137b4c49d7a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:15.234220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:15.252320 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 397/790] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:15.258492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6ab50-58c9-47d6-8008-db1f1368b22d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:15.324531 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:15.406003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5c895-c34d-4bec-8f5a-f137b4c49d7a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:15.459379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6ab50-58c9-47d6-8008-db1f1368b22d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:15.471237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f05e2d9-5485-4cf4-bf3a-dbe950e0f83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:15.472280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05e2d9-5485-4cf4-bf3a-dbe950e0f83d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:15.473170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05e2d9-5485-4cf4-bf3a-dbe950e0f83d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:15.473646 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f05e2d9-5485-4cf4-bf3a-dbe950e0f83d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:15.489759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05e2d9-5485-4cf4-bf3a-dbe950e0f83d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:15.491349 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f05e2d9-5485-4cf4-bf3a-dbe950e0f83d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:15.504626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1801ac-1729-4eca-9d0e-d8cd63d5c461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:15.505683 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1801ac-1729-4eca-9d0e-d8cd63d5c461 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:15.506185 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1801ac-1729-4eca-9d0e-d8cd63d5c461 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:15.506687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1801ac-1729-4eca-9d0e-d8cd63d5c461 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4f05e2d9-5485-4cf4-bf3a-dbe950e0f83d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:15.507091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:15.507964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:15.508879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:15.508879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:15.508879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:15.508879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:15.508879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 394/791] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:15.509799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9e3e5-20b1-498b-9229-30c33ab04566 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:15.510078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9e3e5-20b1-498b-9229-30c33ab04566 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:15.510350 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9e3e5-20b1-498b-9229-30c33ab04566 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:15.521119 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 395/792] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:15 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:15.521119 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9e3e5-20b1-498b-9229-30c33ab04566 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:15.526205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1801ac-1729-4eca-9d0e-d8cd63d5c461 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:15.528006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1801ac-1729-4eca-9d0e-d8cd63d5c461 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2b1801ac-1729-4eca-9d0e-d8cd63d5c461 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:15.542020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:15.543126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:15.544096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:15.544096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:15.544096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:15.544096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:15.544096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:15.544096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:15.544096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:15.544096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:15.544096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:01:15.544096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:15.546905 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 398/793] 10.222.0.100 () {56 vars in 1098 bytes} [Mon Aug 1 15:01:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:15.548454 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68523f67-5832-47fa-b507-46a729608562 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:15.548950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68523f67-5832-47fa-b507-46a729608562 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:15.549419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68523f67-5832-47fa-b507-46a729608562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:15.574098 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-68523f67-5832-47fa-b507-46a729608562 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:15.575909 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-68523f67-5832-47fa-b507-46a729608562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:15.632750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9e3e5-20b1-498b-9229-30c33ab04566 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:15.639851 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:15.642806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:15.663098 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 399/794] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:15 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:15.664335 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68523f67-5832-47fa-b507-46a729608562 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:15.674675 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f834d80-4ca9-4b2c-8da8-12c2179e93f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:15.675542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f834d80-4ca9-4b2c-8da8-12c2179e93f1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:15.675942 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f834d80-4ca9-4b2c-8da8-12c2179e93f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:15.676340 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f834d80-4ca9-4b2c-8da8-12c2179e93f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:15.677256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f834d80-4ca9-4b2c-8da8-12c2179e93f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:15.677708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f834d80-4ca9-4b2c-8da8-12c2179e93f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:15.679650 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f834d80-4ca9-4b2c-8da8-12c2179e93f1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:15.693492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 396/795] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:15.701907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f834d80-4ca9-4b2c-8da8-12c2179e93f1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:15.815035 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68523f67-5832-47fa-b507-46a729608562 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:15.820049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f834d80-4ca9-4b2c-8da8-12c2179e93f1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:15.856648 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636cf3a-a10d-4991-b68f-d754f61f27e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:15.857222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9a4e7-d445-4658-84a5-7cdaea159d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:15.857963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636cf3a-a10d-4991-b68f-d754f61f27e4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:15.858121 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9a4e7-d445-4658-84a5-7cdaea159d10 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:15.858427 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9a4e7-d445-4658-84a5-7cdaea159d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:15.858719 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636cf3a-a10d-4991-b68f-d754f61f27e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:15.858935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9a4e7-d445-4658-84a5-7cdaea159d10 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:15.859278 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636cf3a-a10d-4991-b68f-d754f61f27e4 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:15.860445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636cf3a-a10d-4991-b68f-d754f61f27e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:15.860900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636cf3a-a10d-4991-b68f-d754f61f27e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:15.862754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636cf3a-a10d-4991-b68f-d754f61f27e4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:15.867695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9a4e7-d445-4658-84a5-7cdaea159d10 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:15.869277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9a4e7-d445-4658-84a5-7cdaea159d10 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a8e9a4e7-d445-4658-84a5-7cdaea159d10 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:15.884028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:15.885032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:15.885990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:15.885990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:15.885990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:15.885990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:15.885990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:15.885990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:15.885990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:15.885990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:15.885990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:01:15.885990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:15.885990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 397/796] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:01:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:15.886581 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9a4e7-d445-4658-84a5-7cdaea159d10 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:15.887293 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 400/797] 10.222.0.100 () {56 vars in 1138 bytes} [Mon Aug 1 15:01:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:15.889378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636cf3a-a10d-4991-b68f-d754f61f27e4 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:15.921227 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:15.952075 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:16.009329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9a4e7-d445-4658-84a5-7cdaea159d10 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:16.013522 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636cf3a-a10d-4991-b68f-d754f61f27e4 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:16.048390 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a06d8-221d-43e6-9a1f-d7a3c2cad88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:16.049048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a06d8-221d-43e6-9a1f-d7a3c2cad88f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:16.049333 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a06d8-221d-43e6-9a1f-d7a3c2cad88f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:16.049616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a06d8-221d-43e6-9a1f-d7a3c2cad88f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:16.050251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a06d8-221d-43e6-9a1f-d7a3c2cad88f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:16.051752 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a06d8-221d-43e6-9a1f-d7a3c2cad88f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:16.055904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed82d1-bc44-4636-bbe5-76b84b17ab04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:16.056731 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed82d1-bc44-4636-bbe5-76b84b17ab04 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:16.057154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed82d1-bc44-4636-bbe5-76b84b17ab04 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:16.057568 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed82d1-bc44-4636-bbe5-76b84b17ab04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6d2a06d8-221d-43e6-9a1f-d7a3c2cad88f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:16.061223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:16.062364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 15:01:16.063519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:16.067283 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 398/798] 10.222.0.100 () {56 vars in 1134 bytes} [Mon Aug 1 15:01:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:16.068272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a06d8-221d-43e6-9a1f-d7a3c2cad88f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:16.142877 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed82d1-bc44-4636-bbe5-76b84b17ab04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:16.144276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ed82d1-bc44-4636-bbe5-76b84b17ab04 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:16.170678 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:16.180892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a06d8-221d-43e6-9a1f-d7a3c2cad88f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:16.192999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:16.216758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbec4e-93c6-4bfc-892c-cc8170e5769e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:16.217383 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbec4e-93c6-4bfc-892c-cc8170e5769e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:16.217667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbec4e-93c6-4bfc-892c-cc8170e5769e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:16.217940 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbec4e-93c6-4bfc-892c-cc8170e5769e None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:16.218767 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbec4e-93c6-4bfc-892c-cc8170e5769e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:16.219074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbec4e-93c6-4bfc-892c-cc8170e5769e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:16.220596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbec4e-93c6-4bfc-892c-cc8170e5769e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:16.239175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 399/799] 10.222.0.100 () {56 vars in 1140 bytes} [Mon Aug 1 15:01:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:16.240476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be698dd0-dd61-4e96-8a75-62fc713a4a14 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:16.240768 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be698dd0-dd61-4e96-8a75-62fc713a4a14 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:16.241048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be698dd0-dd61-4e96-8a75-62fc713a4a14 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:16.242277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 400/800] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:16.243062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be698dd0-dd61-4e96-8a75-62fc713a4a14 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:16.266962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 401/801] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:15 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 15:01:16.266962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f063a-fee4-4049-b7ea-71dc38347afa None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:16.266962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f063a-fee4-4049-b7ea-71dc38347afa None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:16.266962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f063a-fee4-4049-b7ea-71dc38347afa None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:16.266962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 402/802] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:16.266962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f063a-fee4-4049-b7ea-71dc38347afa None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:16.354274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be698dd0-dd61-4e96-8a75-62fc713a4a14 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:16.376962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f063a-fee4-4049-b7ea-71dc38347afa None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:16.389728 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ae522-0550-42d8-996b-16b73d48632b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:16.390348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ae522-0550-42d8-996b-16b73d48632b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:16.390630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ae522-0550-42d8-996b-16b73d48632b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:16.390928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ae522-0550-42d8-996b-16b73d48632b None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:16.391608 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ae522-0550-42d8-996b-16b73d48632b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:16.391910 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ae522-0550-42d8-996b-16b73d48632b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:16.393285 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ae522-0550-42d8-996b-16b73d48632b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:16.397971 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 401/803] 10.222.0.100 () {56 vars in 1129 bytes} [Mon Aug 1 15:01:16 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:16.399268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97636d-3d28-4706-848d-fbf88a8a5456 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:16.399549 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97636d-3d28-4706-848d-fbf88a8a5456 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:16.399915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97636d-3d28-4706-848d-fbf88a8a5456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:16.413580 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d76069-5486-4d87-a2c9-a67d51cc982a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:16.414217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d76069-5486-4d87-a2c9-a67d51cc982a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:16.414500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d76069-5486-4d87-a2c9-a67d51cc982a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:16.414797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d76069-5486-4d87-a2c9-a67d51cc982a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:16.418480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6d97636d-3d28-4706-848d-fbf88a8a5456 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:16.419744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6d97636d-3d28-4706-848d-fbf88a8a5456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:16.427780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d76069-5486-4d87-a2c9-a67d51cc982a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:16.429237 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d76069-5486-4d87-a2c9-a67d51cc982a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:16.432495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:16.436577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-c0d76069-5486-4d87-a2c9-a67d51cc982a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:16.445870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:16.446827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:16.447741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:16.447741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:16.447741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:16.447741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:16.447741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 403/804] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:16.448018 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d76069-5486-4d87-a2c9-a67d51cc982a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:16.497124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 402/805] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:16 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:16.498455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3090062-51a3-4e40-bed1-b98ab0946ffc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:16.498801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3090062-51a3-4e40-bed1-b98ab0946ffc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:16.499055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3090062-51a3-4e40-bed1-b98ab0946ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:16.520723 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b3090062-51a3-4e40-bed1-b98ab0946ffc None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:16.521941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b3090062-51a3-4e40-bed1-b98ab0946ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:16.561063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d76069-5486-4d87-a2c9-a67d51cc982a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:16.594188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 403/806] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:16 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:16.595076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3090062-51a3-4e40-bed1-b98ab0946ffc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:16.595331 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfefa9-6d52-42bd-b9b2-3ae54864ff95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:16.595985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfefa9-6d52-42bd-b9b2-3ae54864ff95 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:16.596267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfefa9-6d52-42bd-b9b2-3ae54864ff95 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:16.596541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfefa9-6d52-42bd-b9b2-3ae54864ff95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:16.618146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfefa9-6d52-42bd-b9b2-3ae54864ff95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:16.619677 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bfefa9-6d52-42bd-b9b2-3ae54864ff95 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-29bfefa9-6d52-42bd-b9b2-3ae54864ff95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:16.634630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:16.635807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:16.636747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:16.636747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:16.636747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:16.636747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:16.636747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 404/807] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:16.636747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfefa9-6d52-42bd-b9b2-3ae54864ff95 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:16.676113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:16.688405 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:16.702361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3090062-51a3-4e40-bed1-b98ab0946ffc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:16.738535 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2830e8-4e9d-4b87-858f-561a5955e77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:16.739188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2830e8-4e9d-4b87-858f-561a5955e77c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:16.739500 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2830e8-4e9d-4b87-858f-561a5955e77c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:16.739832 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2830e8-4e9d-4b87-858f-561a5955e77c None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:16.740473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2830e8-4e9d-4b87-858f-561a5955e77c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:16.740665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfefa9-6d52-42bd-b9b2-3ae54864ff95 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:16.740767 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2830e8-4e9d-4b87-858f-561a5955e77c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:16.742155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2830e8-4e9d-4b87-858f-561a5955e77c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:16.747245 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 404/808] 10.222.0.100 () {56 vars in 1129 bytes} [Mon Aug 1 15:01:16 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:16.748063 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2830e8-4e9d-4b87-858f-561a5955e77c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:16.783151 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ff750-13e1-4ea3-a6fb-719300e79914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:16.783794 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ff750-13e1-4ea3-a6fb-719300e79914 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:16.784090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ff750-13e1-4ea3-a6fb-719300e79914 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:16.784356 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2ff750-13e1-4ea3-a6fb-719300e79914 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:16.785067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ff750-13e1-4ea3-a6fb-719300e79914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:16.785381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ff750-13e1-4ea3-a6fb-719300e79914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:16.787437 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2ff750-13e1-4ea3-a6fb-719300e79914 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:16.799958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 405/809] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:16.800823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ff750-13e1-4ea3-a6fb-719300e79914 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:16.856563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2830e8-4e9d-4b87-858f-561a5955e77c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:16.891116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e530f3-4de6-4c83-ae61-11b24f544317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:16.891758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e530f3-4de6-4c83-ae61-11b24f544317 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:16.892039 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e530f3-4de6-4c83-ae61-11b24f544317 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:16.892310 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e530f3-4de6-4c83-ae61-11b24f544317 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:16.900119 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:16.906214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e530f3-4de6-4c83-ae61-11b24f544317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:16.907008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2ff750-13e1-4ea3-a6fb-719300e79914 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-59e530f3-4de6-4c83-ae61-11b24f544317 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:16.911177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:16.912831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:16.913724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:16.913724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:16.913724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:16.913724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:16.913724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:16.913724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:16.913724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:16.913724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:16.913724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:16.913724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:16.913724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:01:16.913724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:16.913724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 405/810] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:01:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:16.914436 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e530f3-4de6-4c83-ae61-11b24f544317 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:16.921913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:16.946423 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9af61-0c69-40e1-b93f-b07b0d7e4f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:16.947021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9af61-0c69-40e1-b93f-b07b0d7e4f3b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:16.947317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9af61-0c69-40e1-b93f-b07b0d7e4f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:16.947599 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9af61-0c69-40e1-b93f-b07b0d7e4f3b None admin] PATH_INFO: `/v3/services/network` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:16.948234 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae9af61-0c69-40e1-b93f-b07b0d7e4f3b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:16.949607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae9af61-0c69-40e1-b93f-b07b0d7e4f3b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-cae9af61-0c69-40e1-b93f-b07b0d7e4f3b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:16.959280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:16.960481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:16.961342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 406/811] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:16 2022] GET /identity/v3/services/network => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:16.962144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9af61-0c69-40e1-b93f-b07b0d7e4f3b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:17.025095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e530f3-4de6-4c83-ae61-11b24f544317 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:17.059061 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10fdfb0-d6dd-48f9-9df8-4a4c86e3c3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:17.059691 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fdfb0-d6dd-48f9-9df8-4a4c86e3c3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:17.059984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fdfb0-d6dd-48f9-9df8-4a4c86e3c3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:17.060330 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10fdfb0-d6dd-48f9-9df8-4a4c86e3c3b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:17.061029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fdfb0-d6dd-48f9-9df8-4a4c86e3c3b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:17.061332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fdfb0-d6dd-48f9-9df8-4a4c86e3c3b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:17.062729 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10fdfb0-d6dd-48f9-9df8-4a4c86e3c3b4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:17.067995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9af61-0c69-40e1-b93f-b07b0d7e4f3b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:17.075483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 406/812] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:17.076527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10fdfb0-d6dd-48f9-9df8-4a4c86e3c3b4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:17.107676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5ba13-833b-4c37-aeab-d76c077d19f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:17.108294 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5ba13-833b-4c37-aeab-d76c077d19f3 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:17.108572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5ba13-833b-4c37-aeab-d76c077d19f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:17.108853 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5ba13-833b-4c37-aeab-d76c077d19f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:17.109564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5ba13-833b-4c37-aeab-d76c077d19f3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:17.110967 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5ba13-833b-4c37-aeab-d76c077d19f3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:17.134817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 407/813] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:01:16 2022] POST /identity/v3/endpoints => generated 348 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:01:17.135665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5ba13-833b-4c37-aeab-d76c077d19f3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:17.149515 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:17.170762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:17.207273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10fdfb0-d6dd-48f9-9df8-4a4c86e3c3b4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:17.254287 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b0a22-6453-4f30-b53c-47fb76e7b7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:17.255510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b0a22-6453-4f30-b53c-47fb76e7b7eb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:17.255794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b0a22-6453-4f30-b53c-47fb76e7b7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:17.256066 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b0a22-6453-4f30-b53c-47fb76e7b7eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:17.263897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b0a22-6453-4f30-b53c-47fb76e7b7eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:17.265368 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b0a22-6453-4f30-b53c-47fb76e7b7eb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:17.266081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5ba13-833b-4c37-aeab-d76c077d19f3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-115b0a22-6453-4f30-b53c-47fb76e7b7eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:17.278424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:17.279467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:17.280365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:17.280365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:17.280365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:17.280365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:17.280365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:17.280365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:17.280365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:17.280365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:17.280365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 407/814] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:17.280851 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b0a22-6453-4f30-b53c-47fb76e7b7eb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:17.309841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d3484-6d2b-4424-bafe-a06a40b3fe25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:17.310472 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d3484-6d2b-4424-bafe-a06a40b3fe25 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:17.310798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d3484-6d2b-4424-bafe-a06a40b3fe25 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:17.311073 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d3484-6d2b-4424-bafe-a06a40b3fe25 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:17.311797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d3484-6d2b-4424-bafe-a06a40b3fe25 None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:17.312106 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d3484-6d2b-4424-bafe-a06a40b3fe25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:17.313548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d3484-6d2b-4424-bafe-a06a40b3fe25 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:17.323873 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 408/815] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:17 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:17.324738 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d3484-6d2b-4424-bafe-a06a40b3fe25 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:17.404509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b0a22-6453-4f30-b53c-47fb76e7b7eb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:17.419254 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:17.446766 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:17.447825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f15bd4-ed8e-4635-b70b-377dfd63f209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:17.448812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f15bd4-ed8e-4635-b70b-377dfd63f209 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:17.449104 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f15bd4-ed8e-4635-b70b-377dfd63f209 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:17.449484 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f15bd4-ed8e-4635-b70b-377dfd63f209 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:17.450295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f15bd4-ed8e-4635-b70b-377dfd63f209 None admin] RBAC: Adding query filter params (name=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:17.451027 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f15bd4-ed8e-4635-b70b-377dfd63f209 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:17.452695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f15bd4-ed8e-4635-b70b-377dfd63f209 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:17.456900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d3484-6d2b-4424-bafe-a06a40b3fe25 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:17.459372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 408/816] 10.222.0.100 () {56 vars in 1117 bytes} [Mon Aug 1 15:01:17 2022] GET /identity/v3/services?name=network => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:17.460237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f15bd4-ed8e-4635-b70b-377dfd63f209 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:17.500691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba563255-2027-4019-b68f-d9dd9b5e36d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:17.501341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba563255-2027-4019-b68f-d9dd9b5e36d5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:17.501632 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba563255-2027-4019-b68f-d9dd9b5e36d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:17.501910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba563255-2027-4019-b68f-d9dd9b5e36d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:17.510157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba563255-2027-4019-b68f-d9dd9b5e36d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:17.512214 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba563255-2027-4019-b68f-d9dd9b5e36d5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-ba563255-2027-4019-b68f-d9dd9b5e36d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:17.524665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:17.525652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:17.526537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:17.526537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:17.526537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:17.526537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:17.526537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:17.526537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:17.526537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:17.526537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:17.526537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 409/817] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:17.527070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba563255-2027-4019-b68f-d9dd9b5e36d5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:17.586053 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f15bd4-ed8e-4635-b70b-377dfd63f209 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:17.627967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93f58d-7a2e-4e97-a162-4ffde4c6dce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:17.628598 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93f58d-7a2e-4e97-a162-4ffde4c6dce4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:17.628887 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93f58d-7a2e-4e97-a162-4ffde4c6dce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:17.629170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93f58d-7a2e-4e97-a162-4ffde4c6dce4 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:17.629986 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93f58d-7a2e-4e97-a162-4ffde4c6dce4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:17.630895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93f58d-7a2e-4e97-a162-4ffde4c6dce4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:17.632354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93f58d-7a2e-4e97-a162-4ffde4c6dce4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:17.648187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 409/818] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:17.649115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93f58d-7a2e-4e97-a162-4ffde4c6dce4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:17.660161 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba563255-2027-4019-b68f-d9dd9b5e36d5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:17.695946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:17.703855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86540c-7f54-4675-a7bf-6fc129309bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:17.704906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86540c-7f54-4675-a7bf-6fc129309bef None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:17.705353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86540c-7f54-4675-a7bf-6fc129309bef None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:17.705644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86540c-7f54-4675-a7bf-6fc129309bef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:17.719783 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86540c-7f54-4675-a7bf-6fc129309bef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:17.720698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86540c-7f54-4675-a7bf-6fc129309bef None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-db86540c-7f54-4675-a7bf-6fc129309bef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:17.738282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:17.739869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:17.740774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:17.740774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:17.740774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:17.740774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:17.740774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 410/819] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:17.741065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86540c-7f54-4675-a7bf-6fc129309bef None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:17.747373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:17.776604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93f58d-7a2e-4e97-a162-4ffde4c6dce4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:17.821049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2482a2-7762-47f8-b579-f3e89ffa7334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:17.821687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2482a2-7762-47f8-b579-f3e89ffa7334 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:17.821976 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2482a2-7762-47f8-b579-f3e89ffa7334 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:17.822247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2482a2-7762-47f8-b579-f3e89ffa7334 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:17.823084 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2482a2-7762-47f8-b579-f3e89ffa7334 None admin] RBAC: Adding query filter params (type=network) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:17.823401 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2482a2-7762-47f8-b579-f3e89ffa7334 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:17.824786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2482a2-7762-47f8-b579-f3e89ffa7334 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:17.830306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 410/820] 10.222.0.100 () {56 vars in 1117 bytes} [Mon Aug 1 15:01:17 2022] GET /identity/v3/services?type=network => generated 352 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:17.831482 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2482a2-7762-47f8-b579-f3e89ffa7334 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:17.862805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86540c-7f54-4675-a7bf-6fc129309bef None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:17.904149 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d8c67-f2a9-43cb-92a9-e5a7bf4d5bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:17.904775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d8c67-f2a9-43cb-92a9-e5a7bf4d5bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:17.905065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d8c67-f2a9-43cb-92a9-e5a7bf4d5bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:17.905608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d8c67-f2a9-43cb-92a9-e5a7bf4d5bc0 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:17.906099 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d8c67-f2a9-43cb-92a9-e5a7bf4d5bc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:17.906413 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d8c67-f2a9-43cb-92a9-e5a7bf4d5bc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:17.907888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904d8c67-f2a9-43cb-92a9-e5a7bf4d5bc0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:17.922305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 411/821] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:17.923099 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d8c67-f2a9-43cb-92a9-e5a7bf4d5bc0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:17.963269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2482a2-7762-47f8-b579-f3e89ffa7334 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:17.965364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:17.996661 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:18.015911 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84608a54-e8c4-4d2b-aed8-f6648b29fb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:18.016587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608a54-e8c4-4d2b-aed8-f6648b29fb65 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:18.016870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608a54-e8c4-4d2b-aed8-f6648b29fb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:18.017149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84608a54-e8c4-4d2b-aed8-f6648b29fb65 None admin] PATH_INFO: `/v3/projects/d6e147b6607b481591ceb250f7208c16/users/41a399a3cc7446c6a8dad94075851797/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:18.019202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84608a54-e8c4-4d2b-aed8-f6648b29fb65 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6e147b6607b481591ceb250f7208c16, user_id=41a399a3cc7446c6a8dad94075851797, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:18.020681 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84608a54-e8c4-4d2b-aed8-f6648b29fb65 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:18.035917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 411/822] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:01:17 2022] PUT /identity/v3/projects/d6e147b6607b481591ceb250f7208c16/users/41a399a3cc7446c6a8dad94075851797/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:01:18.037267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84608a54-e8c4-4d2b-aed8-f6648b29fb65 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:18.051808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d8c67-f2a9-43cb-92a9-e5a7bf4d5bc0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:18.113843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf440e-8d69-4c74-884d-fa811055b0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:18.115539 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf440e-8d69-4c74-884d-fa811055b0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:18.116097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf440e-8d69-4c74-884d-fa811055b0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:18.116592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf440e-8d69-4c74-884d-fa811055b0f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:18.117931 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf440e-8d69-4c74-884d-fa811055b0f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:18.118474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf440e-8d69-4c74-884d-fa811055b0f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:18.121355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf440e-8d69-4c74-884d-fa811055b0f3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:18.140496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 412/823] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:18.141737 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf440e-8d69-4c74-884d-fa811055b0f3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:18.182644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84608a54-e8c4-4d2b-aed8-f6648b29fb65 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:18.233572 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b63d67-19a4-4851-bfd2-269e9a1d4d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:18.235082 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b63d67-19a4-4851-bfd2-269e9a1d4d0b None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:18.235576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b63d67-19a4-4851-bfd2-269e9a1d4d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:18.236072 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b63d67-19a4-4851-bfd2-269e9a1d4d0b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:18.236943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b63d67-19a4-4851-bfd2-269e9a1d4d0b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:18.238335 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b63d67-19a4-4851-bfd2-269e9a1d4d0b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:18.276085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 412/824] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:01:18 2022] POST /identity/v3/endpoints => generated 329 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:01:18.277595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b63d67-19a4-4851-bfd2-269e9a1d4d0b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:18.286100 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf440e-8d69-4c74-884d-fa811055b0f3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:18.314025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:18.321323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:18.354389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ee18-15f8-4d9b-89ab-9477e1b292b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:18.355329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ee18-15f8-4d9b-89ab-9477e1b292b5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:18.355841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ee18-15f8-4d9b-89ab-9477e1b292b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:18.356286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87ee18-15f8-4d9b-89ab-9477e1b292b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:18.357691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87ee18-15f8-4d9b-89ab-9477e1b292b5 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=ec50ac7c54324884a1b2a6c63d617907) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:18.358305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87ee18-15f8-4d9b-89ab-9477e1b292b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:18.360040 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87ee18-15f8-4d9b-89ab-9477e1b292b5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:18.374230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 413/825] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:18 2022] GET /identity/v3/role_assignments?user.id=ec50ac7c54324884a1b2a6c63d617907&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 610 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:18.417478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b63d67-19a4-4851-bfd2-269e9a1d4d0b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:18.458713 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f429a8e-396c-4610-9d8d-7f12f57bb2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:18.459470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f429a8e-396c-4610-9d8d-7f12f57bb2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:18.459762 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f429a8e-396c-4610-9d8d-7f12f57bb2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:18.460048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f429a8e-396c-4610-9d8d-7f12f57bb2e1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:18.469818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f429a8e-396c-4610-9d8d-7f12f57bb2e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:18.471348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f429a8e-396c-4610-9d8d-7f12f57bb2e1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-2f429a8e-396c-4610-9d8d-7f12f57bb2e1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:18.484745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:18.485759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:18.487018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:18.487018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:18.487018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:18.487018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:18.487018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:18.487018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:18.487018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:18.487018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:18.487018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:01:18.487018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:18.487018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 413/826] 10.222.0.100 () {56 vars in 1122 bytes} [Mon Aug 1 15:01:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:18.493105 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ee18-15f8-4d9b-89ab-9477e1b292b5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:18.661713 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87ee18-15f8-4d9b-89ab-9477e1b292b5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:18.700503 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c527be-0bb2-4e3c-b272-487fb4a236fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:18.701482 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c527be-0bb2-4e3c-b272-487fb4a236fd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:18.702018 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c527be-0bb2-4e3c-b272-487fb4a236fd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:18.702492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c527be-0bb2-4e3c-b272-487fb4a236fd None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:18.703534 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c527be-0bb2-4e3c-b272-487fb4a236fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:18.704051 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c527be-0bb2-4e3c-b272-487fb4a236fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:18.705721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c527be-0bb2-4e3c-b272-487fb4a236fd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:18.727017 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 414/827] 10.222.0.100 () {56 vars in 1154 bytes} [Mon Aug 1 15:01:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:18.738648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f429a8e-396c-4610-9d8d-7f12f57bb2e1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:18.855163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:18.885742 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f429a8e-396c-4610-9d8d-7f12f57bb2e1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:18.931206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c67e81-1f8e-4a53-a277-ed6615141d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:18.931863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c67e81-1f8e-4a53-a277-ed6615141d4e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:18.932143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c67e81-1f8e-4a53-a277-ed6615141d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:18.932415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c67e81-1f8e-4a53-a277-ed6615141d4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:18.944314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c67e81-1f8e-4a53-a277-ed6615141d4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:18.945779 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c67e81-1f8e-4a53-a277-ed6615141d4e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-09c67e81-1f8e-4a53-a277-ed6615141d4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:18.962202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:18.963488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:18.964580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:18.964580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:18.964580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:18.964580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:18.964580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 414/828] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:18.969387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c527be-0bb2-4e3c-b272-487fb4a236fd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:19.015812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:19.080673 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c527be-0bb2-4e3c-b272-487fb4a236fd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:19.122046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca33c5d-9a89-4dbd-9925-029ca883b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:19.122746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca33c5d-9a89-4dbd-9925-029ca883b46d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:19.123042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca33c5d-9a89-4dbd-9925-029ca883b46d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:19.123319 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca33c5d-9a89-4dbd-9925-029ca883b46d None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:19.124060 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca33c5d-9a89-4dbd-9925-029ca883b46d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:19.124375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca33c5d-9a89-4dbd-9925-029ca883b46d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:19.125771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca33c5d-9a89-4dbd-9925-029ca883b46d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:19.138953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 415/829] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:19.144402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c67e81-1f8e-4a53-a277-ed6615141d4e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:19.275470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c67e81-1f8e-4a53-a277-ed6615141d4e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:19.295232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5747c-c1ba-4e73-836e-b9f25bd554d3 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:19.295765 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5747c-c1ba-4e73-836e-b9f25bd554d3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:19.295765 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5747c-c1ba-4e73-836e-b9f25bd554d3 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:19.297366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 416/830] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:01:19.303420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487fc36-7e25-4366-ad2b-1be8cbed1ad8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:19.303698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487fc36-7e25-4366-ad2b-1be8cbed1ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:19.303980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487fc36-7e25-4366-ad2b-1be8cbed1ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:19.311826 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:19.318478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31cb3f-9254-43cc-8fd2-fc50efef0394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:19.319892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31cb3f-9254-43cc-8fd2-fc50efef0394 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:19.320413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31cb3f-9254-43cc-8fd2-fc50efef0394 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:19.320906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31cb3f-9254-43cc-8fd2-fc50efef0394 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:19.323634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4487fc36-7e25-4366-ad2b-1be8cbed1ad8 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:19.324991 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4487fc36-7e25-4366-ad2b-1be8cbed1ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:19.330519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31cb3f-9254-43cc-8fd2-fc50efef0394 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:19.332425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31cb3f-9254-43cc-8fd2-fc50efef0394 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6e31cb3f-9254-43cc-8fd2-fc50efef0394 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:19.356710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:19.358543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:19.360371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:19.360371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:19.360371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:19.360371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:19.360371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:19.360371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:19.360371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:19.360371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:19.361923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 415/831] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:19.369846 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31cb3f-9254-43cc-8fd2-fc50efef0394 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:19.427663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 417/832] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:19 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:19.441985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487fc36-7e25-4366-ad2b-1be8cbed1ad8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:19.480774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:19.505356 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31cb3f-9254-43cc-8fd2-fc50efef0394 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:19.562431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffc594-e4e1-4034-9edf-806f193e80b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:19.563734 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffc594-e4e1-4034-9edf-806f193e80b7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:19.564645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffc594-e4e1-4034-9edf-806f193e80b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:19.565381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dffc594-e4e1-4034-9edf-806f193e80b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:19.566354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffc594-e4e1-4034-9edf-806f193e80b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:19.566913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffc594-e4e1-4034-9edf-806f193e80b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:19.567039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487fc36-7e25-4366-ad2b-1be8cbed1ad8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:19.569127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dffc594-e4e1-4034-9edf-806f193e80b7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:19.592718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 416/833] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:19.593880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffc594-e4e1-4034-9edf-806f193e80b7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:19.610907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51180c-3f71-4a27-b356-8bc0812e306c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:19.611783 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51180c-3f71-4a27-b356-8bc0812e306c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:19.611921 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51180c-3f71-4a27-b356-8bc0812e306c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:19.612110 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51180c-3f71-4a27-b356-8bc0812e306c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:19.612925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51180c-3f71-4a27-b356-8bc0812e306c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:19.615104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51180c-3f71-4a27-b356-8bc0812e306c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-0d51180c-3f71-4a27-b356-8bc0812e306c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:19.625290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:19.626364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:19.627341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:19.627341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:19.627341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:19.627341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:19.627341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:19.627341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:19.627341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:19.627341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:19.627341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:19.627341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:19.627341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:19.627341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:19.627341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:01:19.627341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:19.630977 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 418/834] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:01:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:19.632774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51180c-3f71-4a27-b356-8bc0812e306c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:19.665942 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:19.735284 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dffc594-e4e1-4034-9edf-806f193e80b7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:19.761335 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:19.772233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51180c-3f71-4a27-b356-8bc0812e306c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:19.778008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a402c2e-848e-463b-a738-72fcda566eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:19.778969 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a402c2e-848e-463b-a738-72fcda566eb2 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:19.779344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a402c2e-848e-463b-a738-72fcda566eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:19.779691 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a402c2e-848e-463b-a738-72fcda566eb2 None admin] PATH_INFO: `/v3/projects/d6e147b6607b481591ceb250f7208c16/users/d0723b88d7cd481cb962072f75d835c7/roles/70e0c42f1bdd49178db398637f021bbc` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:19.800935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a402c2e-848e-463b-a738-72fcda566eb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6e147b6607b481591ceb250f7208c16, user_id=d0723b88d7cd481cb962072f75d835c7, role_id=70e0c42f1bdd49178db398637f021bbc)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:19.803791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a402c2e-848e-463b-a738-72fcda566eb2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:19.818126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc6a45-e3b4-4f40-847e-9039748859e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:19.819129 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc6a45-e3b4-4f40-847e-9039748859e7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:19.819630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc6a45-e3b4-4f40-847e-9039748859e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:19.820049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc6a45-e3b4-4f40-847e-9039748859e7 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:19.820979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc6a45-e3b4-4f40-847e-9039748859e7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:19.821432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc6a45-e3b4-4f40-847e-9039748859e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:19.826699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 417/835] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:01:19 2022] PUT /identity/v3/projects/d6e147b6607b481591ceb250f7208c16/users/d0723b88d7cd481cb962072f75d835c7/roles/70e0c42f1bdd49178db398637f021bbc => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:01:19.828650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc6a45-e3b4-4f40-847e-9039748859e7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:19.835210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 419/836] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:01:19 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:19.841456 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a402c2e-848e-463b-a738-72fcda566eb2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:19.981094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:19.986651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a402c2e-848e-463b-a738-72fcda566eb2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:20.031236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26033574-cc5e-4366-9686-73d1c62127c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:20.031888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26033574-cc5e-4366-9686-73d1c62127c5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:20.032226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26033574-cc5e-4366-9686-73d1c62127c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:20.032510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26033574-cc5e-4366-9686-73d1c62127c5 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:20.033207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26033574-cc5e-4366-9686-73d1c62127c5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:20.033517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26033574-cc5e-4366-9686-73d1c62127c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:20.034938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26033574-cc5e-4366-9686-73d1c62127c5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:20.040586 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 418/837] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:01:19 2022] GET /identity/v3/services?type=volumev3 => generated 126 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:20.294269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e597bf-32ab-4f13-bfe8-55ae6f63b15a None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:20.294465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e597bf-32ab-4f13-bfe8-55ae6f63b15a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:20.294797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e597bf-32ab-4f13-bfe8-55ae6f63b15a None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:20.296489 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 420/838] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:20.302104 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0070067-48e5-400f-8da1-f734937303ea None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:20.302579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0070067-48e5-400f-8da1-f734937303ea None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:20.302656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0070067-48e5-400f-8da1-f734937303ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:20.326594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c0070067-48e5-400f-8da1-f734937303ea None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:20.328199 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c0070067-48e5-400f-8da1-f734937303ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:20.408807 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 419/839] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:20 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:20.431922 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e597bf-32ab-4f13-bfe8-55ae6f63b15a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:20.568771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e597bf-32ab-4f13-bfe8-55ae6f63b15a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:20.606057 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a92220-bdc6-4980-a6bb-0d6111520fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:20.606734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a92220-bdc6-4980-a6bb-0d6111520fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:20.607020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a92220-bdc6-4980-a6bb-0d6111520fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:20.607295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a92220-bdc6-4980-a6bb-0d6111520fa5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:20.619930 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a92220-bdc6-4980-a6bb-0d6111520fa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-67a92220-bdc6-4980-a6bb-0d6111520fa5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:20.625266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:20.627033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:20.628274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:20.628274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:20.628274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:20.628274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:20.628274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:20.628274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:20.628274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:20.628274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:20.628274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:20.628274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:20.628274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:20.628274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:20.628274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 421/840] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:01:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:20.633001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0070067-48e5-400f-8da1-f734937303ea None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:20.746062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:20.755315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0070067-48e5-400f-8da1-f734937303ea None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:20.792527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33053ca8-d889-4808-950c-eca6800d44dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:20.793159 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33053ca8-d889-4808-950c-eca6800d44dc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:20.793438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33053ca8-d889-4808-950c-eca6800d44dc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:20.793712 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33053ca8-d889-4808-950c-eca6800d44dc None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:20.794444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33053ca8-d889-4808-950c-eca6800d44dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:20.794778 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33053ca8-d889-4808-950c-eca6800d44dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:20.796181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33053ca8-d889-4808-950c-eca6800d44dc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:20.806086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 420/841] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:01:20 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:20.811596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a92220-bdc6-4980-a6bb-0d6111520fa5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:20.838577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:20.852488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b705e-7fb1-4869-8ee2-b76fbc2aece5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:20.852818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b705e-7fb1-4869-8ee2-b76fbc2aece5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:20.853123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b705e-7fb1-4869-8ee2-b76fbc2aece5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:20.855733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 421/842] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:01:20.860065 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf0f3c-be13-472b-b4da-bd392779f623 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:20.860353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf0f3c-be13-472b-b4da-bd392779f623 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:20.860632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf0f3c-be13-472b-b4da-bd392779f623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:20.881101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-18bf0f3c-be13-472b-b4da-bd392779f623 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:20.882328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-18bf0f3c-be13-472b-b4da-bd392779f623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:20.953722 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a92220-bdc6-4980-a6bb-0d6111520fa5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:20.978811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 422/843] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:20 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:20.993071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1f342-22fa-41b6-80b7-04bffa365907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:20.993728 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1f342-22fa-41b6-80b7-04bffa365907 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:20.994011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1f342-22fa-41b6-80b7-04bffa365907 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:20.994346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1f342-22fa-41b6-80b7-04bffa365907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:20.995484 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf0f3c-be13-472b-b4da-bd392779f623 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:21.008968 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1f342-22fa-41b6-80b7-04bffa365907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:21.010587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1f342-22fa-41b6-80b7-04bffa365907 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-90c1f342-22fa-41b6-80b7-04bffa365907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:21.030636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:21.031868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:21.033029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:21.033029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:21.033029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:21.033029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:21.033029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 422/844] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:21.037672 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1f342-22fa-41b6-80b7-04bffa365907 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:21.095822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:21.116774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf0f3c-be13-472b-b4da-bd392779f623 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:21.151074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:21.165236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8171fc-a60a-4367-b122-7edbeb4a726a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:21.165895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8171fc-a60a-4367-b122-7edbeb4a726a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:21.166175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8171fc-a60a-4367-b122-7edbeb4a726a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:21.166446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8171fc-a60a-4367-b122-7edbeb4a726a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:21.167118 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8171fc-a60a-4367-b122-7edbeb4a726a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:21.168501 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8171fc-a60a-4367-b122-7edbeb4a726a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:21.176167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1f342-22fa-41b6-80b7-04bffa365907 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-fc8171fc-a60a-4367-b122-7edbeb4a726a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:21.178114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:21.179308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:21.180276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 423/845] 10.222.0.100 () {56 vars in 1137 bytes} [Mon Aug 1 15:01:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:21.187981 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8171fc-a60a-4367-b122-7edbeb4a726a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:21.225146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120b6b6-7e37-4cb3-8cc3-158ec59b77c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:21.225807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120b6b6-7e37-4cb3-8cc3-158ec59b77c7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:21.226183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120b6b6-7e37-4cb3-8cc3-158ec59b77c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:21.226704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120b6b6-7e37-4cb3-8cc3-158ec59b77c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:21.227863 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120b6b6-7e37-4cb3-8cc3-158ec59b77c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:21.228329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120b6b6-7e37-4cb3-8cc3-158ec59b77c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:21.229854 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120b6b6-7e37-4cb3-8cc3-158ec59b77c7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:21.250154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 423/846] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:21.256106 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120b6b6-7e37-4cb3-8cc3-158ec59b77c7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:21.339292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8171fc-a60a-4367-b122-7edbeb4a726a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:21.376211 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28050460-d0c3-43d2-ab5c-e0bf5b5a46f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:21.376749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28050460-d0c3-43d2-ab5c-e0bf5b5a46f5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:21.377027 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28050460-d0c3-43d2-ab5c-e0bf5b5a46f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:21.377351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28050460-d0c3-43d2-ab5c-e0bf5b5a46f5 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:21.378033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28050460-d0c3-43d2-ab5c-e0bf5b5a46f5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:21.378338 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28050460-d0c3-43d2-ab5c-e0bf5b5a46f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:21.379797 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28050460-d0c3-43d2-ab5c-e0bf5b5a46f5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:21.389807 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 424/847] 10.222.0.100 () {56 vars in 1131 bytes} [Mon Aug 1 15:01:21 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:21.390748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28050460-d0c3-43d2-ab5c-e0bf5b5a46f5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:21.390974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120b6b6-7e37-4cb3-8cc3-158ec59b77c7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:21.412167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:21.427288 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:21.437917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba836b-4ab7-4ab0-8e48-904296e13ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:21.438564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba836b-4ab7-4ab0-8e48-904296e13ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:21.438868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba836b-4ab7-4ab0-8e48-904296e13ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:21.439139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba836b-4ab7-4ab0-8e48-904296e13ba1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:21.450595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba836b-4ab7-4ab0-8e48-904296e13ba1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:21.452074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba836b-4ab7-4ab0-8e48-904296e13ba1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-6eba836b-4ab7-4ab0-8e48-904296e13ba1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:21.465401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:21.466736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:21.468572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:21.468572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:21.468572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:21.468572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:21.468572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:21.468572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:21.468572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:21.468572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:21.468572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 15:01:21.468572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:21.471644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 424/848] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:01:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:21.472586 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba836b-4ab7-4ab0-8e48-904296e13ba1 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:21.546927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28050460-d0c3-43d2-ab5c-e0bf5b5a46f5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:21.586978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1bab5-de73-435d-aa87-6cbe828c78c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:21.588202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1bab5-de73-435d-aa87-6cbe828c78c6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:21.588669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1bab5-de73-435d-aa87-6cbe828c78c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:21.589134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1bab5-de73-435d-aa87-6cbe828c78c6 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:21.590002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1bab5-de73-435d-aa87-6cbe828c78c6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:21.590501 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1bab5-de73-435d-aa87-6cbe828c78c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:21.592085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1bab5-de73-435d-aa87-6cbe828c78c6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:21.603103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 425/849] 10.222.0.100 () {56 vars in 1131 bytes} [Mon Aug 1 15:01:21 2022] GET /identity/v3/services?type=compute_legacy => generated 132 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:21.605066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba836b-4ab7-4ab0-8e48-904296e13ba1 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:21.648182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e090a61-3dfe-4baa-b54b-b21288a9df7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:21.648743 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e090a61-3dfe-4baa-b54b-b21288a9df7c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:21.649026 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e090a61-3dfe-4baa-b54b-b21288a9df7c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:21.649305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e090a61-3dfe-4baa-b54b-b21288a9df7c None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:21.650130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e090a61-3dfe-4baa-b54b-b21288a9df7c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:21.650445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e090a61-3dfe-4baa-b54b-b21288a9df7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:21.651968 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e090a61-3dfe-4baa-b54b-b21288a9df7c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:21.673857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 425/850] 10.222.0.100 () {56 vars in 1140 bytes} [Mon Aug 1 15:01:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:21.680894 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1bab5-de73-435d-aa87-6cbe828c78c6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:21.776582 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:21.803276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1bab5-de73-435d-aa87-6cbe828c78c6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:21.843783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce47729-65a6-427b-b409-beca2e5d12dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:21.844700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce47729-65a6-427b-b409-beca2e5d12dc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:21.845146 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce47729-65a6-427b-b409-beca2e5d12dc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:21.845636 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce47729-65a6-427b-b409-beca2e5d12dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:21.935726 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce47729-65a6-427b-b409-beca2e5d12dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:21.937113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce47729-65a6-427b-b409-beca2e5d12dc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:22.023358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:22.047905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 426/851] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:21 2022] GET /identity/v3/auth/tokens => generated 1153 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:01:22.053818 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e090a61-3dfe-4baa-b54b-b21288a9df7c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:22.093900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538747cb-dcc4-4b30-8576-959bdf97cb1f None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:22.094407 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538747cb-dcc4-4b30-8576-959bdf97cb1f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:22.094895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538747cb-dcc4-4b30-8576-959bdf97cb1f None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:22.097164 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 427/852] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:22 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:22.103482 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2103da-5745-4753-98f2-3716bd41a6bc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:22.104112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2103da-5745-4753-98f2-3716bd41a6bc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:22.104587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2103da-5745-4753-98f2-3716bd41a6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:22.116521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:22.124815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1d2103da-5745-4753-98f2-3716bd41a6bc None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:22.126485 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1d2103da-5745-4753-98f2-3716bd41a6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:22.174324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e090a61-3dfe-4baa-b54b-b21288a9df7c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:22.211586 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904b48c-d7f0-44ae-b290-9cd03c16d9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:22.212198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904b48c-d7f0-44ae-b290-9cd03c16d9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:22.212753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904b48c-d7f0-44ae-b290-9cd03c16d9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:22.213054 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904b48c-d7f0-44ae-b290-9cd03c16d9a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:22.220089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 428/853] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:22 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:22.225095 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904b48c-d7f0-44ae-b290-9cd03c16d9a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:22.226587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904b48c-d7f0-44ae-b290-9cd03c16d9a3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:22.229025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c82752-86c6-4301-b0ae-6904dce24513 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:22.229315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c82752-86c6-4301-b0ae-6904dce24513 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:22.229590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c82752-86c6-4301-b0ae-6904dce24513 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:22.231239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 429/854] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:22.236566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7b65e-6293-4ac9-b752-a1ec5bcee9f2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:22.236850 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7b65e-6293-4ac9-b752-a1ec5bcee9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:22.237127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7b65e-6293-4ac9-b752-a1ec5bcee9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-5904b48c-d7f0-44ae-b290-9cd03c16d9a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:22.244318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:22.245274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:22.246224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:22.246224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:22.246224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:22.246224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:22.250282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 426/855] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:22.251245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904b48c-d7f0-44ae-b290-9cd03c16d9a3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:22.258552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f9f7b65e-6293-4ac9-b752-a1ec5bcee9f2 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:22.259793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f9f7b65e-6293-4ac9-b752-a1ec5bcee9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:22.344006 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 430/856] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:22 2022] POST /identity/v3/auth/tokens => generated 1153 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:22.344963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7b65e-6293-4ac9-b752-a1ec5bcee9f2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:22.372768 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904b48c-d7f0-44ae-b290-9cd03c16d9a3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:22.374500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:22.392159 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:22.421293 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d86bc-1484-4e7c-a3bf-84f569b36656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:22.422389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d86bc-1484-4e7c-a3bf-84f569b36656 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:22.422738 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d86bc-1484-4e7c-a3bf-84f569b36656 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:22.423074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d86bc-1484-4e7c-a3bf-84f569b36656 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:22.435872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d86bc-1484-4e7c-a3bf-84f569b36656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-149d86bc-1484-4e7c-a3bf-84f569b36656 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:22.443166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:22.444368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:22.445429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:22.445429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:22.445429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:22.445429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:22.445429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:22.445429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:22.445429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:22.445429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:22.445429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:22.445429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:22.445429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:01:22.445429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:22.446214 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 427/857] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:01:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:22.446998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d86bc-1484-4e7c-a3bf-84f569b36656 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:22.467386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7b65e-6293-4ac9-b752-a1ec5bcee9f2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:22.509606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe321c2-7f7c-4088-9bac-31d0c05251ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:22.510421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe321c2-7f7c-4088-9bac-31d0c05251ed None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:22.510843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe321c2-7f7c-4088-9bac-31d0c05251ed None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:22.511275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe321c2-7f7c-4088-9bac-31d0c05251ed None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:22.512028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe321c2-7f7c-4088-9bac-31d0c05251ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:22.512442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe321c2-7f7c-4088-9bac-31d0c05251ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:22.513934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe321c2-7f7c-4088-9bac-31d0c05251ed None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:22.527469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 431/858] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:22.528448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe321c2-7f7c-4088-9bac-31d0c05251ed None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:22.569537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d86bc-1484-4e7c-a3bf-84f569b36656 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:22.609696 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd49aaa-316a-4a34-802f-07624513de9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:22.610591 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd49aaa-316a-4a34-802f-07624513de9f None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:22.611219 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd49aaa-316a-4a34-802f-07624513de9f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:22.612426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd49aaa-316a-4a34-802f-07624513de9f None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:22.613309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd49aaa-316a-4a34-802f-07624513de9f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:22.614926 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd49aaa-316a-4a34-802f-07624513de9f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:22.642729 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 428/859] 10.222.0.100 () {60 vars in 1146 bytes} [Mon Aug 1 15:01:22 2022] POST /identity/v3/services => generated 252 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:01:22.642729 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd49aaa-316a-4a34-802f-07624513de9f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:22.654432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:22.659767 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe321c2-7f7c-4088-9bac-31d0c05251ed None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:22.681573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:22.727176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0c927-f8c6-4e4b-ad0b-3f3fdfbbb0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:22.727864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0c927-f8c6-4e4b-ad0b-3f3fdfbbb0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:22.728156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0c927-f8c6-4e4b-ad0b-3f3fdfbbb0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:22.728481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0c927-f8c6-4e4b-ad0b-3f3fdfbbb0d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:22.729425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0c927-f8c6-4e4b-ad0b-3f3fdfbbb0d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:22.729740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0c927-f8c6-4e4b-ad0b-3f3fdfbbb0d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:22.731388 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0c927-f8c6-4e4b-ad0b-3f3fdfbbb0d8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:22.742370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 432/860] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:01:22 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:22.749948 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0c927-f8c6-4e4b-ad0b-3f3fdfbbb0d8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:22.783988 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd49aaa-316a-4a34-802f-07624513de9f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:22.828958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fcac0-d017-462a-b8f6-23980df22aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:22.829929 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fcac0-d017-462a-b8f6-23980df22aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:22.830380 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fcac0-d017-462a-b8f6-23980df22aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:22.830874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8fcac0-d017-462a-b8f6-23980df22aa1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:22.840814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8fcac0-d017-462a-b8f6-23980df22aa1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:22.842464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8fcac0-d017-462a-b8f6-23980df22aa1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-3d8fcac0-d017-462a-b8f6-23980df22aa1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:22.856704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:22.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:22.859433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:22.859433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:22.859433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:22.859433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:22.859433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:22.859433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:22.859433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:22.859433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:22.859433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 429/861] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:22.868229 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fcac0-d017-462a-b8f6-23980df22aa1 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:22.907637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0c927-f8c6-4e4b-ad0b-3f3fdfbbb0d8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:22.945647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74f1ba-4972-4bdb-a27d-746f37294c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:22.946880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74f1ba-4972-4bdb-a27d-746f37294c42 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:22.947700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74f1ba-4972-4bdb-a27d-746f37294c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:22.948453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74f1ba-4972-4bdb-a27d-746f37294c42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:22.964722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d74f1ba-4972-4bdb-a27d-746f37294c42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:22.966694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d74f1ba-4972-4bdb-a27d-746f37294c42 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:22.979552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-7d74f1ba-4972-4bdb-a27d-746f37294c42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:22.985069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:22.986202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:22.987332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:22.987332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:22.987332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:22.987332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:22.989740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 433/862] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:22.994936 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74f1ba-4972-4bdb-a27d-746f37294c42 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:23.002885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8fcac0-d017-462a-b8f6-23980df22aa1 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:23.008632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:23.039600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a0676-a605-4caf-8f63-d670cc2f59df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:23.040396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a0676-a605-4caf-8f63-d670cc2f59df None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:23.040773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a0676-a605-4caf-8f63-d670cc2f59df None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:23.041154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a0676-a605-4caf-8f63-d670cc2f59df None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:23.042020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a0676-a605-4caf-8f63-d670cc2f59df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:23.042442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a0676-a605-4caf-8f63-d670cc2f59df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:23.044030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a0676-a605-4caf-8f63-d670cc2f59df None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:23.063052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 430/863] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:23.064522 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a0676-a605-4caf-8f63-d670cc2f59df None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:23.134108 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74f1ba-4972-4bdb-a27d-746f37294c42 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:23.193125 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ede1c-ee73-427b-a149-2616235fdc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:23.193961 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ede1c-ee73-427b-a149-2616235fdc9d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:23.194575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ede1c-ee73-427b-a149-2616235fdc9d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:23.194984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1ede1c-ee73-427b-a149-2616235fdc9d None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:23.196430 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ede1c-ee73-427b-a149-2616235fdc9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:23.197340 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ede1c-ee73-427b-a149-2616235fdc9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:23.199113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1ede1c-ee73-427b-a149-2616235fdc9d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:23.206365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a0676-a605-4caf-8f63-d670cc2f59df None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:23.218846 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 434/864] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:23.221466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ede1c-ee73-427b-a149-2616235fdc9d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:23.255816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1141c-c65f-44d3-a461-c406334ed422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:23.256361 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1141c-c65f-44d3-a461-c406334ed422 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:23.256905 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1141c-c65f-44d3-a461-c406334ed422 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:23.257193 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1141c-c65f-44d3-a461-c406334ed422 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:23.258030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1141c-c65f-44d3-a461-c406334ed422 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=41a399a3cc7446c6a8dad94075851797) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:23.258864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1141c-c65f-44d3-a461-c406334ed422 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:23.260623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1141c-c65f-44d3-a461-c406334ed422 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:23.270755 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 431/865] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:23 2022] GET /identity/v3/role_assignments?user.id=41a399a3cc7446c6a8dad94075851797&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 610 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:23.316289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:23.360709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1ede1c-ee73-427b-a149-2616235fdc9d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:23.407470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd03de-5ffb-47c1-8b71-bf3fc5a4da2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:23.408463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd03de-5ffb-47c1-8b71-bf3fc5a4da2d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:23.408963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd03de-5ffb-47c1-8b71-bf3fc5a4da2d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:23.409409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd03de-5ffb-47c1-8b71-bf3fc5a4da2d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:23.418888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd03de-5ffb-47c1-8b71-bf3fc5a4da2d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:23.420651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd03de-5ffb-47c1-8b71-bf3fc5a4da2d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-fbcd03de-5ffb-47c1-8b71-bf3fc5a4da2d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:23.435801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:23.437892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:23.440023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:23.440023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:23.440023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:23.440023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:23.440023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:23.440023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:23.440023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:23.440023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:23.440023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:01:23.440023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:23.443878 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 435/866] 10.222.0.100 () {56 vars in 1122 bytes} [Mon Aug 1 15:01:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:23.454737 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1141c-c65f-44d3-a461-c406334ed422 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:23.479082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:23.590616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1141c-c65f-44d3-a461-c406334ed422 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:23.635658 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad62d6-85c7-4235-99ed-d279b1929e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:23.636297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad62d6-85c7-4235-99ed-d279b1929e7b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:23.636577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad62d6-85c7-4235-99ed-d279b1929e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:23.636851 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad62d6-85c7-4235-99ed-d279b1929e7b None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:23.637661 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad62d6-85c7-4235-99ed-d279b1929e7b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:23.637970 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad62d6-85c7-4235-99ed-d279b1929e7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:23.639457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad62d6-85c7-4235-99ed-d279b1929e7b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:23.661005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 432/867] 10.222.0.100 () {56 vars in 1154 bytes} [Mon Aug 1 15:01:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:23.666876 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd03de-5ffb-47c1-8b71-bf3fc5a4da2d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:23.791971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fcefa-b540-4ece-9c10-86e1053683fe None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:23.792341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fcefa-b540-4ece-9c10-86e1053683fe None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:23.792683 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fcefa-b540-4ece-9c10-86e1053683fe None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:23.794113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 433/868] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:23 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:23.800921 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83e9f4-9c29-4125-97b5-6bda262d7f15 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:23.801186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83e9f4-9c29-4125-97b5-6bda262d7f15 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:23.801409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83e9f4-9c29-4125-97b5-6bda262d7f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:23.806371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd03de-5ffb-47c1-8b71-bf3fc5a4da2d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:23.823270 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-6a83e9f4-9c29-4125-97b5-6bda262d7f15 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:23.824500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6a83e9f4-9c29-4125-97b5-6bda262d7f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:23.841731 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:23.854328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137b3e6-1518-4552-b8b2-bee5cef1a5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:23.855503 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137b3e6-1518-4552-b8b2-bee5cef1a5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:23.856058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137b3e6-1518-4552-b8b2-bee5cef1a5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:23.856341 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f137b3e6-1518-4552-b8b2-bee5cef1a5b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:23.907763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:23.915100 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 434/869] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:23 2022] POST /identity/v3/auth/tokens => generated 1254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:23.935311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83e9f4-9c29-4125-97b5-6bda262d7f15 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:23.956265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137b3e6-1518-4552-b8b2-bee5cef1a5b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:23.958297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f137b3e6-1518-4552-b8b2-bee5cef1a5b4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:24.093172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 436/870] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:23 2022] GET /identity/v3/auth/tokens => generated 1254 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:01:24.094231 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a83e9f4-9c29-4125-97b5-6bda262d7f15 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:24.101280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137b3e6-1518-4552-b8b2-bee5cef1a5b4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:24.130101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:24.137819 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76b085-2dec-4eaf-a989-f06c2a927713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:24.138907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76b085-2dec-4eaf-a989-f06c2a927713 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:24.139500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76b085-2dec-4eaf-a989-f06c2a927713 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:24.140037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76b085-2dec-4eaf-a989-f06c2a927713 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:24.140962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76b085-2dec-4eaf-a989-f06c2a927713 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:24.142548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76b085-2dec-4eaf-a989-f06c2a927713 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:24.172128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 435/871] 10.222.0.100 () {60 vars in 1146 bytes} [Mon Aug 1 15:01:23 2022] POST /identity/v3/services => generated 270 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:01:24.233049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f137b3e6-1518-4552-b8b2-bee5cef1a5b4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:24.285484 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dec59-34c4-4a8d-a6a6-99f84390bf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:24.286510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dec59-34c4-4a8d-a6a6-99f84390bf13 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:24.287060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dec59-34c4-4a8d-a6a6-99f84390bf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:24.287527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dec59-34c4-4a8d-a6a6-99f84390bf13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:24.300658 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1dec59-34c4-4a8d-a6a6-99f84390bf13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:24.304636 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1dec59-34c4-4a8d-a6a6-99f84390bf13 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-be1dec59-34c4-4a8d-a6a6-99f84390bf13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:24.323109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:24.324098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:24.325052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:24.325052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:24.325052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:24.325052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:24.327380 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 437/872] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:24.336874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76b085-2dec-4eaf-a989-f06c2a927713 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:24.400403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:24.473261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76b085-2dec-4eaf-a989-f06c2a927713 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:24.519354 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864da61-c212-428e-a5c8-5d1c8edb8f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:24.520290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864da61-c212-428e-a5c8-5d1c8edb8f87 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:24.520749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864da61-c212-428e-a5c8-5d1c8edb8f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:24.521176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864da61-c212-428e-a5c8-5d1c8edb8f87 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:24.522137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864da61-c212-428e-a5c8-5d1c8edb8f87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:24.522703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864da61-c212-428e-a5c8-5d1c8edb8f87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:24.524245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864da61-c212-428e-a5c8-5d1c8edb8f87 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:24.549050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 436/873] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:24.554760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dec59-34c4-4a8d-a6a6-99f84390bf13 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:24.647432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:24.681018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dec59-34c4-4a8d-a6a6-99f84390bf13 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:24.724921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc3112-5deb-4947-9c62-620010814335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:24.725589 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc3112-5deb-4947-9c62-620010814335 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:24.725875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc3112-5deb-4947-9c62-620010814335 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:24.726153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc3112-5deb-4947-9c62-620010814335 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:24.734351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc3112-5deb-4947-9c62-620010814335 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:24.735854 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc3112-5deb-4947-9c62-620010814335 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-efdc3112-5deb-4947-9c62-620010814335 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:24.750812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:24.752196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:24.753764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:24.753764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:24.753764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:24.753764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:24.753764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:24.753764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:24.753764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:24.753764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:24.753764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 438/874] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:24.758626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864da61-c212-428e-a5c8-5d1c8edb8f87 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:24.874625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864da61-c212-428e-a5c8-5d1c8edb8f87 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:24.891181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:24.913979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58250f-005d-4dec-b82f-9c8649adf69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:24.915151 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58250f-005d-4dec-b82f-9c8649adf69a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:24.915653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58250f-005d-4dec-b82f-9c8649adf69a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:24.916127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58250f-005d-4dec-b82f-9c8649adf69a None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:24.917090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58250f-005d-4dec-b82f-9c8649adf69a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:24.917616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58250f-005d-4dec-b82f-9c8649adf69a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:24.919387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58250f-005d-4dec-b82f-9c8649adf69a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:24.933030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 437/875] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:24.937852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8aff5-9a71-4be7-bf59-1502bd0f1732 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:24.938232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8aff5-9a71-4be7-bf59-1502bd0f1732 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:24.938512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8aff5-9a71-4be7-bf59-1502bd0f1732 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:24.939508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58250f-005d-4dec-b82f-9c8649adf69a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:24.940168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 439/876] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:24.945942 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87250b5-e626-4eb2-a8c7-117bd2d8c664 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:24.946233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87250b5-e626-4eb2-a8c7-117bd2d8c664 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:24.946513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87250b5-e626-4eb2-a8c7-117bd2d8c664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:24.966410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c87250b5-e626-4eb2-a8c7-117bd2d8c664 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:24.967780 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c87250b5-e626-4eb2-a8c7-117bd2d8c664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:25.066758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 440/877] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:24 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:25.079988 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58250f-005d-4dec-b82f-9c8649adf69a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:25.085092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87250b5-e626-4eb2-a8c7-117bd2d8c664 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:25.128512 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca39d7-7b51-48ba-b053-551c59ea64d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:25.129571 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca39d7-7b51-48ba-b053-551c59ea64d3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:25.130578 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca39d7-7b51-48ba-b053-551c59ea64d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:25.131733 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca39d7-7b51-48ba-b053-551c59ea64d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:25.132817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca39d7-7b51-48ba-b053-551c59ea64d3 None admin] RBAC: Adding query filter params (role.id=70e0c42f1bdd49178db398637f021bbc, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=d0723b88d7cd481cb962072f75d835c7) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:25.133318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca39d7-7b51-48ba-b053-551c59ea64d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:25.135518 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca39d7-7b51-48ba-b053-551c59ea64d3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:25.145041 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 438/878] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:24 2022] GET /identity/v3/role_assignments?user.id=d0723b88d7cd481cb962072f75d835c7&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=70e0c42f1bdd49178db398637f021bbc => generated 610 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:25.167923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:25.241075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87250b5-e626-4eb2-a8c7-117bd2d8c664 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:25.288809 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2cf29-6de0-422b-af61-f505809961b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:25.289943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2cf29-6de0-422b-af61-f505809961b7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:25.290449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2cf29-6de0-422b-af61-f505809961b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:25.290991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2cf29-6de0-422b-af61-f505809961b7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:25.291954 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2cf29-6de0-422b-af61-f505809961b7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:25.293617 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2cf29-6de0-422b-af61-f505809961b7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-28a2cf29-6de0-422b-af61-f505809961b7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:25.306552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:25.307671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:25.308573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:25.308573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:25.308573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:25.308573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:25.308573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:25.308573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:25.308573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:25.308573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:25.308573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:25.308573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:25.308573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:25.308573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:25.308573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:01:25.308573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:25.311821 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 441/879] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:01:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:25.320157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca39d7-7b51-48ba-b053-551c59ea64d3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:25.375942 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:25.455716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca39d7-7b51-48ba-b053-551c59ea64d3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:25.499744 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c6624-79d9-49f0-b95c-435e84a6a965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:25.501079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c6624-79d9-49f0-b95c-435e84a6a965 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:25.501370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c6624-79d9-49f0-b95c-435e84a6a965 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:25.501652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c6624-79d9-49f0-b95c-435e84a6a965 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:25.502482 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c6624-79d9-49f0-b95c-435e84a6a965 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:25.502841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c6624-79d9-49f0-b95c-435e84a6a965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:25.504248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c6624-79d9-49f0-b95c-435e84a6a965 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:25.510817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 439/880] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:01:25 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:25.519841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2cf29-6de0-422b-af61-f505809961b7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:25.640167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bf7a6-0199-40ab-b582-6c8584c7fbd9 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:25.640500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bf7a6-0199-40ab-b582-6c8584c7fbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:25.640724 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17bf7a6-0199-40ab-b582-6c8584c7fbd9 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:25.643535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 440/881] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:01:25.648126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc09c2-2eda-459b-8e0b-f6a8b2358693 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:25.648425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc09c2-2eda-459b-8e0b-f6a8b2358693 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:25.648694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc09c2-2eda-459b-8e0b-f6a8b2358693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:25.660597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2cf29-6de0-422b-af61-f505809961b7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:25.669958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a7bc09c2-2eda-459b-8e0b-f6a8b2358693 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:25.671824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a7bc09c2-2eda-459b-8e0b-f6a8b2358693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:25.707194 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:25.717370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1b9b3-0529-433a-a018-1cb1f017b4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:25.718447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1b9b3-0529-433a-a018-1cb1f017b4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:25.719007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1b9b3-0529-433a-a018-1cb1f017b4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:25.719462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d1b9b3-0529-433a-a018-1cb1f017b4d8 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:25.720303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1b9b3-0529-433a-a018-1cb1f017b4d8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:25.720757 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1b9b3-0529-433a-a018-1cb1f017b4d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:25.723431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d1b9b3-0529-433a-a018-1cb1f017b4d8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:25.730935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 442/882] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:01:25 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:25.737680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1b9b3-0529-433a-a018-1cb1f017b4d8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:25.777957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 441/883] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:25 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:25.796928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc09c2-2eda-459b-8e0b-f6a8b2358693 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:25.840687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:25.872739 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d1b9b3-0529-433a-a018-1cb1f017b4d8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:25.920513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a65cf-0680-4182-9f12-3606046865d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:25.922466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a65cf-0680-4182-9f12-3606046865d9 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:25.923153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a65cf-0680-4182-9f12-3606046865d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:25.923640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a65cf-0680-4182-9f12-3606046865d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:25.924621 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a65cf-0680-4182-9f12-3606046865d9 None admin] RBAC: Adding query filter params (interface=public, service_id=444d0cbfb4914c1eba40375fa71eddd9, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:25.924847 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc09c2-2eda-459b-8e0b-f6a8b2358693 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:25.925361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a65cf-0680-4182-9f12-3606046865d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:25.927152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a65cf-0680-4182-9f12-3606046865d9 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:25.933502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 443/884] 10.222.0.100 () {56 vars in 1256 bytes} [Mon Aug 1 15:01:25 2022] GET /identity/v3/endpoints?service_id=444d0cbfb4914c1eba40375fa71eddd9&interface=public®ion_id=RegionOne => generated 195 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:25.942721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a65cf-0680-4182-9f12-3606046865d9 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:25.965272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2b706-bbfa-4ee8-9237-6ad3af09c75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:25.965925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2b706-bbfa-4ee8-9237-6ad3af09c75d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:25.966244 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2b706-bbfa-4ee8-9237-6ad3af09c75d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:25.966518 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e2b706-bbfa-4ee8-9237-6ad3af09c75d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:25.967363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e2b706-bbfa-4ee8-9237-6ad3af09c75d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:25.968750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e2b706-bbfa-4ee8-9237-6ad3af09c75d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-b9e2b706-bbfa-4ee8-9237-6ad3af09c75d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:25.983052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:25.984125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:25.984976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 442/885] 10.222.0.100 () {56 vars in 1131 bytes} [Mon Aug 1 15:01:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:25.991642 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2b706-bbfa-4ee8-9237-6ad3af09c75d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:26.018630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:26.082629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a65cf-0680-4182-9f12-3606046865d9 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:26.117991 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:26.127615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e2b706-bbfa-4ee8-9237-6ad3af09c75d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:26.135453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683e00-fb3b-4684-9db5-76a08a91d150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:26.136353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683e00-fb3b-4684-9db5-76a08a91d150 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:26.136786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683e00-fb3b-4684-9db5-76a08a91d150 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:26.137209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683e00-fb3b-4684-9db5-76a08a91d150 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:26.138036 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6683e00-fb3b-4684-9db5-76a08a91d150 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:26.140391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6683e00-fb3b-4684-9db5-76a08a91d150 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:26.150207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 444/886] 10.222.0.100 () {56 vars in 1092 bytes} [Mon Aug 1 15:01:25 2022] GET /identity/v3/services => generated 1274 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:01:26.173281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc98b32-ff75-4231-a99b-1637fe974eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:26.174205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc98b32-ff75-4231-a99b-1637fe974eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:26.174640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc98b32-ff75-4231-a99b-1637fe974eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:26.175120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc98b32-ff75-4231-a99b-1637fe974eb2 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:26.176022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc98b32-ff75-4231-a99b-1637fe974eb2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:26.176532 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc98b32-ff75-4231-a99b-1637fe974eb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:26.178187 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc98b32-ff75-4231-a99b-1637fe974eb2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:26.184794 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 443/887] 10.222.0.100 () {56 vars in 1127 bytes} [Mon Aug 1 15:01:25 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:26.190558 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683e00-fb3b-4684-9db5-76a08a91d150 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:26.331175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683e00-fb3b-4684-9db5-76a08a91d150 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:26.362355 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:26.377082 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f929b-3724-47d7-a4cb-9d83d4c6ab05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:26.377723 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f929b-3724-47d7-a4cb-9d83d4c6ab05 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:26.378005 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f929b-3724-47d7-a4cb-9d83d4c6ab05 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:26.378283 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f929b-3724-47d7-a4cb-9d83d4c6ab05 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:26.379084 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f929b-3724-47d7-a4cb-9d83d4c6ab05 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:26.379392 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f929b-3724-47d7-a4cb-9d83d4c6ab05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:26.381324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f929b-3724-47d7-a4cb-9d83d4c6ab05 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:26.387188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 445/888] 10.222.0.100 () {56 vars in 1127 bytes} [Mon Aug 1 15:01:26 2022] GET /identity/v3/services?type=object-store => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:26.525973 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796fcfa-fabf-4c35-a1c1-2c094f1e8fdc None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:26.526551 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796fcfa-fabf-4c35-a1c1-2c094f1e8fdc None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:26.526551 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8796fcfa-fabf-4c35-a1c1-2c094f1e8fdc None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:26.528844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 444/889] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:26 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:26.538243 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af9572-127d-4050-939d-445cb8eb0985 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:26.538243 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af9572-127d-4050-939d-445cb8eb0985 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:26.538243 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af9572-127d-4050-939d-445cb8eb0985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:26.561137 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-08af9572-127d-4050-939d-445cb8eb0985 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:26.562385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-08af9572-127d-4050-939d-445cb8eb0985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:26.643343 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 446/890] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:26 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:26.666254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796fcfa-fabf-4c35-a1c1-2c094f1e8fdc None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:26.786860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8796fcfa-fabf-4c35-a1c1-2c094f1e8fdc None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:26.824276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de05f8-da6a-4e6d-ae5e-e20bf2c998ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:26.824939 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de05f8-da6a-4e6d-ae5e-e20bf2c998ef None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:26.825224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de05f8-da6a-4e6d-ae5e-e20bf2c998ef None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:26.825638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de05f8-da6a-4e6d-ae5e-e20bf2c998ef None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:26.826555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de05f8-da6a-4e6d-ae5e-e20bf2c998ef None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:26.828002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de05f8-da6a-4e6d-ae5e-e20bf2c998ef None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-54de05f8-da6a-4e6d-ae5e-e20bf2c998ef None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:26.838762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:26.839764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:26.840646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 445/891] 10.222.0.100 () {56 vars in 1137 bytes} [Mon Aug 1 15:01:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:26.846537 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af9572-127d-4050-939d-445cb8eb0985 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:26.972412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af9572-127d-4050-939d-445cb8eb0985 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:26.992079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:27.017325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354e52f-de63-4e09-b2d2-a9b2877db4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:27.018378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354e52f-de63-4e09-b2d2-a9b2877db4dd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:27.018946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354e52f-de63-4e09-b2d2-a9b2877db4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:27.019419 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354e52f-de63-4e09-b2d2-a9b2877db4dd None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:27.020900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354e52f-de63-4e09-b2d2-a9b2877db4dd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:27.021428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354e52f-de63-4e09-b2d2-a9b2877db4dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:27.023141 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354e52f-de63-4e09-b2d2-a9b2877db4dd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:27.029488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 447/892] 10.222.0.100 () {56 vars in 1131 bytes} [Mon Aug 1 15:01:26 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:27.035067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de05f8-da6a-4e6d-ae5e-e20bf2c998ef None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:27.065047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:27.158121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de05f8-da6a-4e6d-ae5e-e20bf2c998ef None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:27.207387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6719cfd-b564-41fa-8dea-dcf048e525bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:27.208584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6719cfd-b564-41fa-8dea-dcf048e525bd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:27.208863 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6719cfd-b564-41fa-8dea-dcf048e525bd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:27.209135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6719cfd-b564-41fa-8dea-dcf048e525bd None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:27.209830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6719cfd-b564-41fa-8dea-dcf048e525bd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:27.210137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6719cfd-b564-41fa-8dea-dcf048e525bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:27.211560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6719cfd-b564-41fa-8dea-dcf048e525bd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:27.218053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 446/893] 10.222.0.100 () {56 vars in 1131 bytes} [Mon Aug 1 15:01:27 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:27.227281 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354e52f-de63-4e09-b2d2-a9b2877db4dd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:27.356653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354e52f-de63-4e09-b2d2-a9b2877db4dd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:27.396238 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3413a3-b1ce-430f-9762-0fd32cb57b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:27.396876 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3413a3-b1ce-430f-9762-0fd32cb57b8d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:27.397160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3413a3-b1ce-430f-9762-0fd32cb57b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:27.397435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3413a3-b1ce-430f-9762-0fd32cb57b8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:27.398206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3413a3-b1ce-430f-9762-0fd32cb57b8d None admin] RBAC: Adding query filter params (interface=public, service_id=a75d2c15e23443f2ab26673f2a8be855, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:27.398572 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3413a3-b1ce-430f-9762-0fd32cb57b8d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:27.400074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3413a3-b1ce-430f-9762-0fd32cb57b8d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:27.406361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 448/894] 10.222.0.100 () {56 vars in 1256 bytes} [Mon Aug 1 15:01:27 2022] GET /identity/v3/endpoints?service_id=a75d2c15e23443f2ab26673f2a8be855&interface=public®ion_id=RegionOne => generated 195 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:27.412016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6719cfd-b564-41fa-8dea-dcf048e525bd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:27.484900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8432bb-487f-42f8-b226-1cb693c5e9d8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:27.485122 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8432bb-487f-42f8-b226-1cb693c5e9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:27.485373 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8432bb-487f-42f8-b226-1cb693c5e9d8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:27.486984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 449/895] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:27.493241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021fe41-0baa-47bf-9fce-fb154f2d4479 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:27.493584 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021fe41-0baa-47bf-9fce-fb154f2d4479 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:27.493796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021fe41-0baa-47bf-9fce-fb154f2d4479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:27.515805 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5021fe41-0baa-47bf-9fce-fb154f2d4479 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:27.517024 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5021fe41-0baa-47bf-9fce-fb154f2d4479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:27.553512 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:27.568262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:27.572485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6719cfd-b564-41fa-8dea-dcf048e525bd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:27.608007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 450/896] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:27 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:27.624083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963a782-81e7-4ca8-b6c1-c64048bf39ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:27.625055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963a782-81e7-4ca8-b6c1-c64048bf39ed None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:27.625520 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963a782-81e7-4ca8-b6c1-c64048bf39ed None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:27.625944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963a782-81e7-4ca8-b6c1-c64048bf39ed None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:27.626785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963a782-81e7-4ca8-b6c1-c64048bf39ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:27.628322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963a782-81e7-4ca8-b6c1-c64048bf39ed None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:27.634289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021fe41-0baa-47bf-9fce-fb154f2d4479 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:27.641989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 447/897] 10.222.0.100 () {56 vars in 1092 bytes} [Mon Aug 1 15:01:27 2022] GET /identity/v3/services => generated 1274 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:01:27.783657 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021fe41-0baa-47bf-9fce-fb154f2d4479 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:27.827397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9fdb1-73a1-41e3-a3db-cc7411b52fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:27.828068 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9fdb1-73a1-41e3-a3db-cc7411b52fdd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:27.828357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9fdb1-73a1-41e3-a3db-cc7411b52fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:27.828638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9fdb1-73a1-41e3-a3db-cc7411b52fdd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:27.841119 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9fdb1-73a1-41e3-a3db-cc7411b52fdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b8a9fdb1-73a1-41e3-a3db-cc7411b52fdd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:27.846188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:27.847629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:27.849030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:27.849030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:27.849030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:27.849030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:27.849030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:27.849030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:27.849030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:27.849030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:27.849030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:27.849030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:27.849030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:01:27.849030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:27.849030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 451/898] 10.222.0.100 () {56 vars in 1125 bytes} [Mon Aug 1 15:01:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:27.856029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963a782-81e7-4ca8-b6c1-c64048bf39ed None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:27.979345 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963a782-81e7-4ca8-b6c1-c64048bf39ed None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:28.016113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb63954-7dfd-488a-851a-94af0a1071e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:28.016789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb63954-7dfd-488a-851a-94af0a1071e7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:28.017071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb63954-7dfd-488a-851a-94af0a1071e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:28.017398 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb63954-7dfd-488a-851a-94af0a1071e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:28.018194 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb63954-7dfd-488a-851a-94af0a1071e7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:28.018499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb63954-7dfd-488a-851a-94af0a1071e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:28.019960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb63954-7dfd-488a-851a-94af0a1071e7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:28.030253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 448/899] 10.222.0.100 () {56 vars in 1120 bytes} [Mon Aug 1 15:01:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:28.036681 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9fdb1-73a1-41e3-a3db-cc7411b52fdd None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:28.084910 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:28.164776 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9fdb1-73a1-41e3-a3db-cc7411b52fdd None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:28.204930 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7163a-35ea-4b9a-821a-91059637af19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:28.205691 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7163a-35ea-4b9a-821a-91059637af19 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:28.205970 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7163a-35ea-4b9a-821a-91059637af19 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:28.206428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7163a-35ea-4b9a-821a-91059637af19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:28.219036 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7163a-35ea-4b9a-821a-91059637af19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:28.220563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7163a-35ea-4b9a-821a-91059637af19 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6bb7163a-35ea-4b9a-821a-91059637af19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:28.237113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:28.238097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:28.239185 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:28.239185 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:28.239185 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:28.239185 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:28.239185 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 452/900] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:28.245751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb63954-7dfd-488a-851a-94af0a1071e7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:28.267669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:28.377707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb63954-7dfd-488a-851a-94af0a1071e7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:28.422702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d06aa3-2a7b-4cf4-81fd-7280d5c5aab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:28.423718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d06aa3-2a7b-4cf4-81fd-7280d5c5aab0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:28.424235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d06aa3-2a7b-4cf4-81fd-7280d5c5aab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:28.424714 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d06aa3-2a7b-4cf4-81fd-7280d5c5aab0 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:28.425640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d06aa3-2a7b-4cf4-81fd-7280d5c5aab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:28.426152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d06aa3-2a7b-4cf4-81fd-7280d5c5aab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:28.427823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d06aa3-2a7b-4cf4-81fd-7280d5c5aab0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:28.436001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e1765-5cf8-4bce-b8d4-24f00fe009fc None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:28.436323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e1765-5cf8-4bce-b8d4-24f00fe009fc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:28.436553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e1765-5cf8-4bce-b8d4-24f00fe009fc None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:28.438047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 453/901] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:28.445900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a677f0-80d1-4b7f-abc7-f8dbb09d0da4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:28.445900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a677f0-80d1-4b7f-abc7-f8dbb09d0da4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:28.445900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a677f0-80d1-4b7f-abc7-f8dbb09d0da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:28.450843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 449/902] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:28.456258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d06aa3-2a7b-4cf4-81fd-7280d5c5aab0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:28.471618 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-69a677f0-80d1-4b7f-abc7-f8dbb09d0da4 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:28.472978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-69a677f0-80d1-4b7f-abc7-f8dbb09d0da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:28.551310 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:28.579052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 454/903] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:28 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:28.594939 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d06aa3-2a7b-4cf4-81fd-7280d5c5aab0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:28.601476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a677f0-80d1-4b7f-abc7-f8dbb09d0da4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:28.620084 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:28.645743 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2d647-362a-4e5c-9bcc-ee763d993c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:28.647039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2d647-362a-4e5c-9bcc-ee763d993c77 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:28.647332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2d647-362a-4e5c-9bcc-ee763d993c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:28.647619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2d647-362a-4e5c-9bcc-ee763d993c77 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:28.658498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2d647-362a-4e5c-9bcc-ee763d993c77 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:28.660777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2d647-362a-4e5c-9bcc-ee763d993c77 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-65b2d647-362a-4e5c-9bcc-ee763d993c77 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:28.675907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:28.677213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:28.678419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:28.678419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:28.678419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:28.678419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:28.678419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:28.678419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:28.678419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:28.678419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:28.678419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:01:28.678419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:28.678419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 450/904] 10.222.0.100 () {56 vars in 1122 bytes} [Mon Aug 1 15:01:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:28.679099 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e3cec-e267-4ef4-9e06-64d001abbc93 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:28.679099 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e3cec-e267-4ef4-9e06-64d001abbc93 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:28.679260 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e3cec-e267-4ef4-9e06-64d001abbc93 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:28.682805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 451/905] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:28.688620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e3cec-e267-4ef4-9e06-64d001abbc93 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:28.724382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a677f0-80d1-4b7f-abc7-f8dbb09d0da4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:28.772421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d276864-b5e9-4ef7-a077-ce6ae680f5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:28.773205 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d276864-b5e9-4ef7-a077-ce6ae680f5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:28.773490 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d276864-b5e9-4ef7-a077-ce6ae680f5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:28.773765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d276864-b5e9-4ef7-a077-ce6ae680f5c7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:28.774408 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d276864-b5e9-4ef7-a077-ce6ae680f5c7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:28.775854 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d276864-b5e9-4ef7-a077-ce6ae680f5c7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5d276864-b5e9-4ef7-a077-ce6ae680f5c7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:28.785306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:28.787247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:28.788819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 455/906] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:01:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:28.790263 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948996f5-3581-43c3-84f6-eeafd473c1b1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:28.790263 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948996f5-3581-43c3-84f6-eeafd473c1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:28.790263 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948996f5-3581-43c3-84f6-eeafd473c1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:28.818170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-948996f5-3581-43c3-84f6-eeafd473c1b1 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:28.820224 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-948996f5-3581-43c3-84f6-eeafd473c1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:28.825702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e3cec-e267-4ef4-9e06-64d001abbc93 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:28.882137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d08c9-3baf-4fac-a38f-5c57d2bdd476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:28.887065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d08c9-3baf-4fac-a38f-5c57d2bdd476 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:28.887065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d08c9-3baf-4fac-a38f-5c57d2bdd476 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:28.887065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d08c9-3baf-4fac-a38f-5c57d2bdd476 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:28.887065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d08c9-3baf-4fac-a38f-5c57d2bdd476 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:28.887065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d08c9-3baf-4fac-a38f-5c57d2bdd476 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:28.888573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d08c9-3baf-4fac-a38f-5c57d2bdd476 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:28.891289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:28.913540 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 452/907] 10.222.0.100 () {56 vars in 1154 bytes} [Mon Aug 1 15:01:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:28.914423 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d08c9-3baf-4fac-a38f-5c57d2bdd476 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:28.934927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 456/908] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:28 2022] POST /identity/v3/auth/tokens => generated 1364 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:28.936553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948996f5-3581-43c3-84f6-eeafd473c1b1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:28.955287 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:29.041372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d08c9-3baf-4fac-a38f-5c57d2bdd476 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:29.061997 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948996f5-3581-43c3-84f6-eeafd473c1b1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:29.081626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe589cd-ac5c-4dd0-b609-85d1e4418bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:29.082252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe589cd-ac5c-4dd0-b609-85d1e4418bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:29.082575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe589cd-ac5c-4dd0-b609-85d1e4418bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:29.082882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe589cd-ac5c-4dd0-b609-85d1e4418bb8 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:29.083573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe589cd-ac5c-4dd0-b609-85d1e4418bb8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:29.083882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe589cd-ac5c-4dd0-b609-85d1e4418bb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:29.085313 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe589cd-ac5c-4dd0-b609-85d1e4418bb8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:29.101212 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 453/909] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:01:28 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:29.101212 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe589cd-ac5c-4dd0-b609-85d1e4418bb8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:29.112358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a244f0-01b0-4986-986f-29b534469514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:29.113009 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a244f0-01b0-4986-986f-29b534469514 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:29.113293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a244f0-01b0-4986-986f-29b534469514 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:29.113573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a244f0-01b0-4986-986f-29b534469514 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:29.174148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:29.203584 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a244f0-01b0-4986-986f-29b534469514 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:29.206123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a244f0-01b0-4986-986f-29b534469514 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:29.226725 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:29.244355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe589cd-ac5c-4dd0-b609-85d1e4418bb8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:29.279713 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0335272-9db4-4fb6-9469-559e4b02f0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:29.280336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0335272-9db4-4fb6-9469-559e4b02f0ae None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:29.280619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0335272-9db4-4fb6-9469-559e4b02f0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:29.280894 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0335272-9db4-4fb6-9469-559e4b02f0ae None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:29.281594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0335272-9db4-4fb6-9469-559e4b02f0ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:29.283007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0335272-9db4-4fb6-9469-559e4b02f0ae None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:29.301414 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 454/910] 10.222.0.100 () {60 vars in 1146 bytes} [Mon Aug 1 15:01:29 2022] POST /identity/v3/services => generated 242 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:01:29.302905 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0335272-9db4-4fb6-9469-559e4b02f0ae None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:29.373231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 457/911] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:28 2022] GET /identity/v3/auth/tokens => generated 1364 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:01:29.379375 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a244f0-01b0-4986-986f-29b534469514 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:29.489744 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0335272-9db4-4fb6-9469-559e4b02f0ae None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:29.509644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:29.515490 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a244f0-01b0-4986-986f-29b534469514 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:29.530494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:29.543797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb8e4c-2a86-45d0-8f54-5420c2e25454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:29.545548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb8e4c-2a86-45d0-8f54-5420c2e25454 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:29.546332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb8e4c-2a86-45d0-8f54-5420c2e25454 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:29.546868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb8e4c-2a86-45d0-8f54-5420c2e25454 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:29.548050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb8e4c-2a86-45d0-8f54-5420c2e25454 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:29.549056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb8e4c-2a86-45d0-8f54-5420c2e25454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:29.551294 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb8e4c-2a86-45d0-8f54-5420c2e25454 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:29.559859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 455/912] 10.222.0.100 () {56 vars in 1119 bytes} [Mon Aug 1 15:01:29 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:29.560733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18704bc-9ab7-4891-8086-65e19189a33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:29.561407 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18704bc-9ab7-4891-8086-65e19189a33f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:29.561695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18704bc-9ab7-4891-8086-65e19189a33f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:29.562052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18704bc-9ab7-4891-8086-65e19189a33f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:29.566021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb8e4c-2a86-45d0-8f54-5420c2e25454 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:29.576828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18704bc-9ab7-4891-8086-65e19189a33f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:29.578324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18704bc-9ab7-4891-8086-65e19189a33f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e18704bc-9ab7-4891-8086-65e19189a33f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:29.600411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:29.601965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:29.603187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:29.603187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:29.603187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:29.603187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:29.603187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 458/913] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:29.608554 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18704bc-9ab7-4891-8086-65e19189a33f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:29.711806 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb8e4c-2a86-45d0-8f54-5420c2e25454 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:29.745618 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18704bc-9ab7-4891-8086-65e19189a33f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:29.752651 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37590e09-ba53-4928-9cff-1690e87c61a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:29.753400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37590e09-ba53-4928-9cff-1690e87c61a5 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:29.753692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37590e09-ba53-4928-9cff-1690e87c61a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:29.753972 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37590e09-ba53-4928-9cff-1690e87c61a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:29.755395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37590e09-ba53-4928-9cff-1690e87c61a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:29.756895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37590e09-ba53-4928-9cff-1690e87c61a5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:29.783157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 456/914] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:01:29 2022] POST /identity/v3/endpoints => generated 348 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:01:29.801104 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97a4d3-b4ac-4ded-9bca-1cee6f9c8ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:29.802357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97a4d3-b4ac-4ded-9bca-1cee6f9c8ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:29.806694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97a4d3-b4ac-4ded-9bca-1cee6f9c8ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:29.806694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97a4d3-b4ac-4ded-9bca-1cee6f9c8ab4 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:29.806694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97a4d3-b4ac-4ded-9bca-1cee6f9c8ab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:29.806694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97a4d3-b4ac-4ded-9bca-1cee6f9c8ab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:29.806694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97a4d3-b4ac-4ded-9bca-1cee6f9c8ab4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:29.822287 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 459/915] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:29.827908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37590e09-ba53-4928-9cff-1690e87c61a5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:29.881355 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ddc88-d552-4226-ad20-61d5b3b4b59e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:29.881641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ddc88-d552-4226-ad20-61d5b3b4b59e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:29.882028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ddc88-d552-4226-ad20-61d5b3b4b59e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:29.883837 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:29.884155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 460/916] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:29 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:29.892452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94336f62-9f59-4e5b-a5c9-d1e7fab170d5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:29.893715 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94336f62-9f59-4e5b-a5c9-d1e7fab170d5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:29.894957 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94336f62-9f59-4e5b-a5c9-d1e7fab170d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:29.917511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-94336f62-9f59-4e5b-a5c9-d1e7fab170d5 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:29.919194 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-94336f62-9f59-4e5b-a5c9-d1e7fab170d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:29.937058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:29.966749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37590e09-ba53-4928-9cff-1690e87c61a5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:30.014789 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 461/917] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:29 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:30.023689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922da35e-f9cb-4b1e-a936-8adbbe8c7300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:30.024381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922da35e-f9cb-4b1e-a936-8adbbe8c7300 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:30.024665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922da35e-f9cb-4b1e-a936-8adbbe8c7300 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:30.024941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922da35e-f9cb-4b1e-a936-8adbbe8c7300 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:30.035117 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922da35e-f9cb-4b1e-a936-8adbbe8c7300 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:30.036720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94336f62-9f59-4e5b-a5c9-d1e7fab170d5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:30.039978 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922da35e-f9cb-4b1e-a936-8adbbe8c7300 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-922da35e-f9cb-4b1e-a936-8adbbe8c7300 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:30.055474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:30.056458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:30.057372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:30.057372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:30.057372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:30.057372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:30.057372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:30.057372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:30.057372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:30.057372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:30.057372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 457/918] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:30.064406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922da35e-f9cb-4b1e-a936-8adbbe8c7300 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:30.167513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94336f62-9f59-4e5b-a5c9-d1e7fab170d5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:30.169963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:30.180463 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922da35e-f9cb-4b1e-a936-8adbbe8c7300 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:30.209428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739abb0-afa2-4752-9cfb-5e94dc3acbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:30.210093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739abb0-afa2-4752-9cfb-5e94dc3acbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:30.210378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739abb0-afa2-4752-9cfb-5e94dc3acbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:30.210654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739abb0-afa2-4752-9cfb-5e94dc3acbb8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:30.211326 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739abb0-afa2-4752-9cfb-5e94dc3acbb8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:30.213238 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739abb0-afa2-4752-9cfb-5e94dc3acbb8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:30.218375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d8869-7795-431f-9b32-3b4a14e1a8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:30.219131 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d8869-7795-431f-9b32-3b4a14e1a8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:30.219476 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d8869-7795-431f-9b32-3b4a14e1a8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:30.219822 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d8869-7795-431f-9b32-3b4a14e1a8d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:30.220703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d8869-7795-431f-9b32-3b4a14e1a8d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:30.221104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d8869-7795-431f-9b32-3b4a14e1a8d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:30.222645 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d8869-7795-431f-9b32-3b4a14e1a8d5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e739abb0-afa2-4752-9cfb-5e94dc3acbb8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:30.227466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:30.228434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 15:01:30.229309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:30.233924 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 462/919] 10.222.0.100 () {56 vars in 1137 bytes} [Mon Aug 1 15:01:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:30.235362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739abb0-afa2-4752-9cfb-5e94dc3acbb8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:30.242707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 458/920] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:30.243779 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d8869-7795-431f-9b32-3b4a14e1a8d5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:30.252908 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:30.354935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739abb0-afa2-4752-9cfb-5e94dc3acbb8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:30.363519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d8869-7795-431f-9b32-3b4a14e1a8d5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:30.400643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f3012-3ce6-41fd-b9e3-182b4f875ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:30.401269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f3012-3ce6-41fd-b9e3-182b4f875ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:30.401552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f3012-3ce6-41fd-b9e3-182b4f875ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:30.401721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a3712-8193-4006-adc7-17f70524e1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:30.401832 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f3012-3ce6-41fd-b9e3-182b4f875ad9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:30.402335 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a3712-8193-4006-adc7-17f70524e1ae None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:30.402612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f3012-3ce6-41fd-b9e3-182b4f875ad9 None admin] RBAC: Adding query filter params (role.id=46f67f665d95448798ec8aef267e78fb, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=d0723b88d7cd481cb962072f75d835c7) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:30.402734 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a3712-8193-4006-adc7-17f70524e1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:30.402933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a3712-8193-4006-adc7-17f70524e1ae None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:30.403102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f3012-3ce6-41fd-b9e3-182b4f875ad9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:30.403882 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a3712-8193-4006-adc7-17f70524e1ae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:30.404238 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a3712-8193-4006-adc7-17f70524e1ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:30.404402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f3012-3ce6-41fd-b9e3-182b4f875ad9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:30.405676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a3712-8193-4006-adc7-17f70524e1ae None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:30.416686 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 463/921] 10.222.0.100 () {56 vars in 1131 bytes} [Mon Aug 1 15:01:30 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:30.418152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a3712-8193-4006-adc7-17f70524e1ae None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:30.419112 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 459/922] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:30 2022] GET /identity/v3/role_assignments?user.id=d0723b88d7cd481cb962072f75d835c7&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=46f67f665d95448798ec8aef267e78fb => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:30.519183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:30.543247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a3712-8193-4006-adc7-17f70524e1ae None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:30.580891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446a61f-f6e6-4a19-a8f3-c13a4564f363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:30.581536 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446a61f-f6e6-4a19-a8f3-c13a4564f363 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:30.581817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446a61f-f6e6-4a19-a8f3-c13a4564f363 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:30.582093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446a61f-f6e6-4a19-a8f3-c13a4564f363 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:30.583498 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446a61f-f6e6-4a19-a8f3-c13a4564f363 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:30.583809 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446a61f-f6e6-4a19-a8f3-c13a4564f363 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:30.585254 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446a61f-f6e6-4a19-a8f3-c13a4564f363 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:30.596435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 464/923] 10.222.0.100 () {56 vars in 1131 bytes} [Mon Aug 1 15:01:30 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:30.597872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f3012-3ce6-41fd-b9e3-182b4f875ad9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:30.619676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:30.714354 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f3012-3ce6-41fd-b9e3-182b4f875ad9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:30.750135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ade73-2183-4046-98e1-8aa1e9e33e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:30.751168 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ade73-2183-4046-98e1-8aa1e9e33e60 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:30.751625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ade73-2183-4046-98e1-8aa1e9e33e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:30.752075 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ade73-2183-4046-98e1-8aa1e9e33e60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:30.753036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ade73-2183-4046-98e1-8aa1e9e33e60 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:30.755249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ade73-2183-4046-98e1-8aa1e9e33e60 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:30.786783 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 460/924] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:01:30 2022] POST /identity/v3/endpoints => generated 349 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:01:31.528050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75923c58-e4bb-4ab0-8910-d6d216dc183f None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:31.531331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75923c58-e4bb-4ab0-8910-d6d216dc183f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:31.531331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75923c58-e4bb-4ab0-8910-d6d216dc183f None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:31.531331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 465/925] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:31 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:31.540103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c2233-5cbe-477b-9a9c-c68b3cd2565e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:31.540571 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c2233-5cbe-477b-9a9c-c68b3cd2565e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:31.543674 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c2233-5cbe-477b-9a9c-c68b3cd2565e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:31.562541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c66c2233-5cbe-477b-9a9c-c68b3cd2565e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:31.563784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c66c2233-5cbe-477b-9a9c-c68b3cd2565e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:31.645041 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:31.651742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 461/926] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:31 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:31.674443 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75923c58-e4bb-4ab0-8910-d6d216dc183f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:31.791953 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75923c58-e4bb-4ab0-8910-d6d216dc183f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:31.828760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586b6c2-ca04-4ec3-b555-0c24c7ab69b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:31.829469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586b6c2-ca04-4ec3-b555-0c24c7ab69b5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:31.829756 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586b6c2-ca04-4ec3-b555-0c24c7ab69b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:31.830032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586b6c2-ca04-4ec3-b555-0c24c7ab69b5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:31.830688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586b6c2-ca04-4ec3-b555-0c24c7ab69b5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:31.832081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586b6c2-ca04-4ec3-b555-0c24c7ab69b5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4586b6c2-ca04-4ec3-b555-0c24c7ab69b5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:31.842047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:31.843196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:31.844200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:31.844200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:31.844200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:31.844200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:31.844200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:31.844200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:31.844200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:31.844200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:31.844200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:31.844200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:31.844200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:31.844200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:31.844200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:31.844200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:31.850698 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 466/927] 10.222.0.100 () {56 vars in 1131 bytes} [Mon Aug 1 15:01:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:31.855818 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c2233-5cbe-477b-9a9c-c68b3cd2565e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:31.969802 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c2233-5cbe-477b-9a9c-c68b3cd2565e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:32.011342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4241ac-7b12-43fb-9439-265da345ad3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:32.011974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4241ac-7b12-43fb-9439-265da345ad3e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.012251 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4241ac-7b12-43fb-9439-265da345ad3e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.012711 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4241ac-7b12-43fb-9439-265da345ad3e None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.013511 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4241ac-7b12-43fb-9439-265da345ad3e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:32.013821 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4241ac-7b12-43fb-9439-265da345ad3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:32.015244 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4241ac-7b12-43fb-9439-265da345ad3e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:32.021212 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 462/928] 10.222.0.100 () {56 vars in 1127 bytes} [Mon Aug 1 15:01:31 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:32.028075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586b6c2-ca04-4ec3-b555-0c24c7ab69b5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:32.039050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:32.057400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078964d-570c-4a1c-ac53-ef907ec0db8b None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.057727 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078964d-570c-4a1c-ac53-ef907ec0db8b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.058276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078964d-570c-4a1c-ac53-ef907ec0db8b None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.059555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 463/929] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:32.065298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5f891-915d-416a-920a-37369a01ad9a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.065581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5f891-915d-416a-920a-37369a01ad9a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.065908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e5f891-915d-416a-920a-37369a01ad9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.085943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c7e5f891-915d-416a-920a-37369a01ad9a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:32.087200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c7e5f891-915d-416a-920a-37369a01ad9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:32.152797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:32.165845 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586b6c2-ca04-4ec3-b555-0c24c7ab69b5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:32.170969 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 464/930] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:32 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:32.190485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5f891-915d-416a-920a-37369a01ad9a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:32.208900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a53bc-ed82-471d-9e17-fb87a4072a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:32.209533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a53bc-ed82-471d-9e17-fb87a4072a14 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.209811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a53bc-ed82-471d-9e17-fb87a4072a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.210355 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a53bc-ed82-471d-9e17-fb87a4072a14 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.211468 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a53bc-ed82-471d-9e17-fb87a4072a14 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:32.211814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a53bc-ed82-471d-9e17-fb87a4072a14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:32.213526 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a53bc-ed82-471d-9e17-fb87a4072a14 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:32.222188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 467/931] 10.222.0.100 () {56 vars in 1127 bytes} [Mon Aug 1 15:01:32 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:32.228357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a53bc-ed82-471d-9e17-fb87a4072a14 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:32.314150 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e5f891-915d-416a-920a-37369a01ad9a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:32.315127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:32.355862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a53bc-ed82-471d-9e17-fb87a4072a14 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:32.356211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4de40f-a280-4e5d-bf9e-1fbec17beb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:32.356515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4de40f-a280-4e5d-bf9e-1fbec17beb6f None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.356838 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4de40f-a280-4e5d-bf9e-1fbec17beb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.357113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4de40f-a280-4e5d-bf9e-1fbec17beb6f None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.357863 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4de40f-a280-4e5d-bf9e-1fbec17beb6f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:32.359536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4de40f-a280-4e5d-bf9e-1fbec17beb6f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:32.362199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: WARNING keystone.server.flask.common [None req-2b4de40f-a280-4e5d-bf9e-1fbec17beb6f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 15:01:32.385897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.sql.core [None req-2b4de40f-a280-4e5d-bf9e-1fbec17beb6f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 15:01:32.385897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 15:01:32.385897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [parameters: {'id': 'd811785ee8584b7db0b3c3b60902471f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 15:01:32.385897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80359) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 15:01:32.397844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc789e1-8752-45bc-a165-8033b70567c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:32.398530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc789e1-8752-45bc-a165-8033b70567c2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.398968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc789e1-8752-45bc-a165-8033b70567c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.399385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc789e1-8752-45bc-a165-8033b70567c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.400369 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc789e1-8752-45bc-a165-8033b70567c2 None admin] RBAC: Adding query filter params (interface=public, service_id=3557d4234b5c458696e6edbf790b3e98, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:32.400776 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc789e1-8752-45bc-a165-8033b70567c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:32.402307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc789e1-8752-45bc-a165-8033b70567c2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:32.409480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2b4de40f-a280-4e5d-bf9e-1fbec17beb6f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d811785ee8584b7db0b3c3b60902471f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d811785ee8584b7db0b3c3b60902471f. Aug 01 15:01:32.409480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:32.409480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:01:32.409480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:01:32.409480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:01:32.409480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:01:32.409480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:01:32.409480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:01:32.409480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:01:32.409480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:01:32.409480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:01:32.409480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:01:32.409480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result.read() Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 01 15:01:32.410375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application yield resource Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise self.value Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application session.commit() Aug 01 15:01:32.412079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.session.flush() Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application compat.raise_( Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 15:01:32.413006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 15:01:32.414145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result.read() Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:01:32.415011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [parameters: {'id': 'd811785ee8584b7db0b3c3b60902471f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:32.415899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:32.416868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:32.418012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:32.418012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 01 15:01:32.418012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 01 15:01:32.418012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:32.418012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:32.418012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 01 15:01:32.418012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 01 15:01:32.418012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d811785ee8584b7db0b3c3b60902471f. Aug 01 15:01:32.418012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:32.418637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 468/933] 10.222.0.100 () {56 vars in 1256 bytes} [Mon Aug 1 15:01:32 2022] GET /identity/v3/endpoints?service_id=3557d4234b5c458696e6edbf790b3e98&interface=public®ion_id=RegionOne => generated 195 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:32.422876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 465/933] 10.222.0.100 () {60 vars in 1145 bytes} [Mon Aug 1 15:01:32 2022] POST /identity/v3/projects => generated 262 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:32.422876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4de40f-a280-4e5d-bf9e-1fbec17beb6f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:32.423043 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc789e1-8752-45bc-a165-8033b70567c2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:32.492277 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:32.558217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4de40f-a280-4e5d-bf9e-1fbec17beb6f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:32.569711 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc789e1-8752-45bc-a165-8033b70567c2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:32.604589 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:32.605974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1abef-f09b-449f-a711-b4ff242bead3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:32.606718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1abef-f09b-449f-a711-b4ff242bead3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.606929 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1abef-f09b-449f-a711-b4ff242bead3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.607188 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1abef-f09b-449f-a711-b4ff242bead3 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.607848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1abef-f09b-449f-a711-b4ff242bead3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:32.609773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a1abef-f09b-449f-a711-b4ff242bead3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:32.610787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383aa34-3ab9-44a1-9937-30f3d1c347e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:32.611435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383aa34-3ab9-44a1-9937-30f3d1c347e2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.611718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383aa34-3ab9-44a1-9937-30f3d1c347e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.611994 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383aa34-3ab9-44a1-9937-30f3d1c347e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.619325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383aa34-3ab9-44a1-9937-30f3d1c347e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:32.619682 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 466/934] 10.222.0.100 () {56 vars in 1092 bytes} [Mon Aug 1 15:01:32 2022] GET /identity/v3/services => generated 1504 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:01:32.620788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383aa34-3ab9-44a1-9937-30f3d1c347e2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:32.621565 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66a139-752c-4149-9a6d-97cb6af2721d None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.621921 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66a139-752c-4149-9a6d-97cb6af2721d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.622260 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66a139-752c-4149-9a6d-97cb6af2721d None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.626789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 467/935] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:32.629670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f5f5f-38cf-4af8-9ff3-1dfc93ba76ee None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.630173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f5f5f-38cf-4af8-9ff3-1dfc93ba76ee None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.630607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f5f5f-38cf-4af8-9ff3-1dfc93ba76ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9383aa34-3ab9-44a1-9937-30f3d1c347e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:32.641418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:32.642611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:32.643796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:32.643796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:32.643796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:32.643796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:32.643796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:32.643796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:32.643796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:32.643796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:32.646103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 469/936] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:32.652149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383aa34-3ab9-44a1-9937-30f3d1c347e2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:32.658455 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f50f5f5f-38cf-4af8-9ff3-1dfc93ba76ee None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:32.660107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f50f5f5f-38cf-4af8-9ff3-1dfc93ba76ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:32.770704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 468/937] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:32 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:32.792057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383aa34-3ab9-44a1-9937-30f3d1c347e2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:32.796726 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f5f5f-38cf-4af8-9ff3-1dfc93ba76ee None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:32.844684 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d622e7-2c2a-42e8-965c-43792c75b9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:32.845323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d622e7-2c2a-42e8-965c-43792c75b9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.845605 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d622e7-2c2a-42e8-965c-43792c75b9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.845880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d622e7-2c2a-42e8-965c-43792c75b9b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.846757 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d622e7-2c2a-42e8-965c-43792c75b9b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:32.847719 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d622e7-2c2a-42e8-965c-43792c75b9b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:32.849851 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d622e7-2c2a-42e8-965c-43792c75b9b7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:32.864797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:32.871136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 470/938] 10.222.0.100 () {56 vars in 1117 bytes} [Mon Aug 1 15:01:32 2022] GET /identity/v3/projects?name=service => generated 413 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:32.936245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f5f5f-38cf-4af8-9ff3-1dfc93ba76ee None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:32.978714 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a853b2c-4ffa-421e-aa62-a1c3c9817ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:32.979645 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a853b2c-4ffa-421e-aa62-a1c3c9817ecd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.980074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a853b2c-4ffa-421e-aa62-a1c3c9817ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.980520 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a853b2c-4ffa-421e-aa62-a1c3c9817ecd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.986015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb6e6b-6901-488d-b8a0-b149c15e4623 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.986301 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb6e6b-6901-488d-b8a0-b149c15e4623 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.986594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb6e6b-6901-488d-b8a0-b149c15e4623 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.988348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 471/939] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:32.994102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4241a6b-73b2-4adb-addc-ae0a48a85a9b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:32.994338 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4241a6b-73b2-4adb-addc-ae0a48a85a9b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:32.994611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4241a6b-73b2-4adb-addc-ae0a48a85a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:32.995266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a853b2c-4ffa-421e-aa62-a1c3c9817ecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-6a853b2c-4ffa-421e-aa62-a1c3c9817ecd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:33.000530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:33.001734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:33.002807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:33.002807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:33.002807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:33.002807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:33.002807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:33.002807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:33.002807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:33.002807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:33.002807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:33.002807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:33.002807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:01:33.002807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:33.004810 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 469/940] 10.222.0.100 () {56 vars in 1125 bytes} [Mon Aug 1 15:01:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:33.012312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a853b2c-4ffa-421e-aa62-a1c3c9817ecd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:33.026354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d4241a6b-73b2-4adb-addc-ae0a48a85a9b None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:33.028085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d4241a6b-73b2-4adb-addc-ae0a48a85a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:33.099842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:33.126410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 472/941] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:32 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:33.149877 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4241a6b-73b2-4adb-addc-ae0a48a85a9b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:33.152486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a853b2c-4ffa-421e-aa62-a1c3c9817ecd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:33.155217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:33.201566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de46a5f-4463-4609-8ef4-8ff078e2f241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:33.202688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46a5f-4463-4609-8ef4-8ff078e2f241 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:33.203145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46a5f-4463-4609-8ef4-8ff078e2f241 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:33.203725 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de46a5f-4463-4609-8ef4-8ff078e2f241 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:33.204469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de46a5f-4463-4609-8ef4-8ff078e2f241 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:33.204775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de46a5f-4463-4609-8ef4-8ff078e2f241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:33.207005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de46a5f-4463-4609-8ef4-8ff078e2f241 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:33.219838 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 470/942] 10.222.0.100 () {56 vars in 1120 bytes} [Mon Aug 1 15:01:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:33.225741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de46a5f-4463-4609-8ef4-8ff078e2f241 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:33.276229 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4241a6b-73b2-4adb-addc-ae0a48a85a9b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:33.323664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1746b-d2bf-4c04-b16f-82a25b885920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:33.324821 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1746b-d2bf-4c04-b16f-82a25b885920 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:33.325431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1746b-d2bf-4c04-b16f-82a25b885920 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:33.325898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1746b-d2bf-4c04-b16f-82a25b885920 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:33.326557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1746b-d2bf-4c04-b16f-82a25b885920 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:33.328760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1746b-d2bf-4c04-b16f-82a25b885920 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-42c1746b-d2bf-4c04-b16f-82a25b885920 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:33.341400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:33.342747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:33.343839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 473/943] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:33.351870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1746b-d2bf-4c04-b16f-82a25b885920 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:33.357896 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de46a5f-4463-4609-8ef4-8ff078e2f241 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:33.406370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ccd09-0bd4-43cf-b874-e7ca81af58ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:33.407148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ccd09-0bd4-43cf-b874-e7ca81af58ae None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:33.407501 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ccd09-0bd4-43cf-b874-e7ca81af58ae None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:33.407917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ccd09-0bd4-43cf-b874-e7ca81af58ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:33.414162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:33.423073 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ccd09-0bd4-43cf-b874-e7ca81af58ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:33.424982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ccd09-0bd4-43cf-b874-e7ca81af58ae None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-699ccd09-0bd4-43cf-b874-e7ca81af58ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:33.442176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:33.443186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:33.444074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:33.444074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:33.444074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:33.444074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:33.446123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 471/944] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:33.453609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ccd09-0bd4-43cf-b874-e7ca81af58ae None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:33.476376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:33.494328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1746b-d2bf-4c04-b16f-82a25b885920 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:33.532782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd39426-b4d1-4995-9d4b-4820370ce8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:33.533413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd39426-b4d1-4995-9d4b-4820370ce8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:33.533698 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd39426-b4d1-4995-9d4b-4820370ce8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:33.533974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd39426-b4d1-4995-9d4b-4820370ce8c0 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:33.534697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd39426-b4d1-4995-9d4b-4820370ce8c0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:33.535000 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd39426-b4d1-4995-9d4b-4820370ce8c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:33.537014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd39426-b4d1-4995-9d4b-4820370ce8c0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:33.548198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 474/945] 10.222.0.100 () {56 vars in 1117 bytes} [Mon Aug 1 15:01:33 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:33.553888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd39426-b4d1-4995-9d4b-4820370ce8c0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:33.594386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ccd09-0bd4-43cf-b874-e7ca81af58ae None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:33.631435 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b168a-0d56-450d-a563-5c2f08ca0b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:33.631984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b168a-0d56-450d-a563-5c2f08ca0b16 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:33.632261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b168a-0d56-450d-a563-5c2f08ca0b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:33.632533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b168a-0d56-450d-a563-5c2f08ca0b16 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:33.633229 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b168a-0d56-450d-a563-5c2f08ca0b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:33.633540 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b168a-0d56-450d-a563-5c2f08ca0b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:33.634996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520b168a-0d56-450d-a563-5c2f08ca0b16 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:33.648969 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 472/946] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:33.657713 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b168a-0d56-450d-a563-5c2f08ca0b16 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:33.686045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd39426-b4d1-4995-9d4b-4820370ce8c0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:33.700594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:33.729533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c4964-86e9-4ed6-8883-b94bae88dbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:33.730433 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c4964-86e9-4ed6-8883-b94bae88dbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:33.730752 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c4964-86e9-4ed6-8883-b94bae88dbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:33.731036 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c4964-86e9-4ed6-8883-b94bae88dbd3 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:33.731898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c4964-86e9-4ed6-8883-b94bae88dbd3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:33.732210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c4964-86e9-4ed6-8883-b94bae88dbd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:33.734302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c4964-86e9-4ed6-8883-b94bae88dbd3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:33.740707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 475/947] 10.222.0.100 () {56 vars in 1117 bytes} [Mon Aug 1 15:01:33 2022] GET /identity/v3/services?type=compute => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:33.779050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:33.795575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b168a-0d56-450d-a563-5c2f08ca0b16 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:33.838073 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4d5ff-8d78-43f1-adcc-225f7b4544e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:33.838769 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4d5ff-8d78-43f1-adcc-225f7b4544e3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:33.839056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4d5ff-8d78-43f1-adcc-225f7b4544e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:33.839331 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4d5ff-8d78-43f1-adcc-225f7b4544e3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:33.851598 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4d5ff-8d78-43f1-adcc-225f7b4544e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:33.853118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4d5ff-8d78-43f1-adcc-225f7b4544e3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-5dc4d5ff-8d78-43f1-adcc-225f7b4544e3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:33.866952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:33.868460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:33.869433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:33.869433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:33.869433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:33.869433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:33.869433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:33.869433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:33.869433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:33.869433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:33.869433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:01:33.869433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:33.869433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 473/948] 10.222.0.100 () {56 vars in 1122 bytes} [Mon Aug 1 15:01:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:33.874869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c4964-86e9-4ed6-8883-b94bae88dbd3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:33.998323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c4964-86e9-4ed6-8883-b94bae88dbd3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:34.037238 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d949e-66b8-4a9e-8b7c-294adb72e813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:34.038239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d949e-66b8-4a9e-8b7c-294adb72e813 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:34.038802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d949e-66b8-4a9e-8b7c-294adb72e813 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:34.039312 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d949e-66b8-4a9e-8b7c-294adb72e813 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:34.040345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d949e-66b8-4a9e-8b7c-294adb72e813 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:34.040873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d949e-66b8-4a9e-8b7c-294adb72e813 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:34.042523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d949e-66b8-4a9e-8b7c-294adb72e813 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:34.062912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 476/949] 10.222.0.100 () {56 vars in 1154 bytes} [Mon Aug 1 15:01:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:34.068921 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4d5ff-8d78-43f1-adcc-225f7b4544e3 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:34.231660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4d5ff-8d78-43f1-adcc-225f7b4544e3 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:34.271592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc14ed3-9319-466a-95da-757ad696447b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:34.272167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc14ed3-9319-466a-95da-757ad696447b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:34.272445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc14ed3-9319-466a-95da-757ad696447b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:34.272797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc14ed3-9319-466a-95da-757ad696447b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:34.275946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:34.287411 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc14ed3-9319-466a-95da-757ad696447b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:34.288875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc14ed3-9319-466a-95da-757ad696447b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-5dc14ed3-9319-466a-95da-757ad696447b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:34.304655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:34.305643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:34.306543 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:34.306543 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:34.306543 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:34.306543 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:34.306543 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 474/950] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:34.312480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d949e-66b8-4a9e-8b7c-294adb72e813 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:34.376533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:34.563060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d949e-66b8-4a9e-8b7c-294adb72e813 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:34.656428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b18a7-3082-4a66-9c41-0c313d559317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:34.657075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b18a7-3082-4a66-9c41-0c313d559317 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:34.657361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b18a7-3082-4a66-9c41-0c313d559317 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:34.657639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b18a7-3082-4a66-9c41-0c313d559317 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:34.658350 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b18a7-3082-4a66-9c41-0c313d559317 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:34.658726 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b18a7-3082-4a66-9c41-0c313d559317 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:34.660592 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b18a7-3082-4a66-9c41-0c313d559317 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:34.676062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 477/951] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:34.683791 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc14ed3-9319-466a-95da-757ad696447b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:34.826054 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc14ed3-9319-466a-95da-757ad696447b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:34.865277 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974367b-7e44-4b5a-9a44-397c426d11b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:34.866201 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974367b-7e44-4b5a-9a44-397c426d11b2 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:34.866710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974367b-7e44-4b5a-9a44-397c426d11b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:34.867143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974367b-7e44-4b5a-9a44-397c426d11b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:34.876716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974367b-7e44-4b5a-9a44-397c426d11b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:34.878196 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974367b-7e44-4b5a-9a44-397c426d11b2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:34.884644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833cf00-26e3-40ca-b93f-510f89c8bf16 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:34.885337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833cf00-26e3-40ca-b93f-510f89c8bf16 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:34.885337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833cf00-26e3-40ca-b93f-510f89c8bf16 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:34.888034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 478/952] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:34.892326 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c70936-cabe-4e98-a753-f556b051a39e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:34.892611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c70936-cabe-4e98-a753-f556b051a39e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:34.892884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c70936-cabe-4e98-a753-f556b051a39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-a974367b-7e44-4b5a-9a44-397c426d11b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:34.893399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:34.895538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:34.896917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:34.896917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:34.896917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:34.896917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:34.896917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:34.896917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:34.896917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:34.896917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:34.898824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 475/953] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:34.900825 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974367b-7e44-4b5a-9a44-397c426d11b2 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:34.918595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-31c70936-cabe-4e98-a753-f556b051a39e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:34.919836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-31c70936-cabe-4e98-a753-f556b051a39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:34.966363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:35.009431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 479/954] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:34 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:35.028597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c70936-cabe-4e98-a753-f556b051a39e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:35.036628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974367b-7e44-4b5a-9a44-397c426d11b2 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:35.054372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:35.079013 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d559d-6260-440e-83e5-1f27657c31f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:35.080033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d559d-6260-440e-83e5-1f27657c31f9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:35.080575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d559d-6260-440e-83e5-1f27657c31f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:35.081062 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d559d-6260-440e-83e5-1f27657c31f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:35.082120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d559d-6260-440e-83e5-1f27657c31f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:35.082772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d559d-6260-440e-83e5-1f27657c31f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:35.084750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d559d-6260-440e-83e5-1f27657c31f9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:35.100688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 476/955] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:35.106446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d559d-6260-440e-83e5-1f27657c31f9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:35.144049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c70936-cabe-4e98-a753-f556b051a39e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:35.181220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97605bfc-6ffe-462d-925a-b6041a91cd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:35.181888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97605bfc-6ffe-462d-925a-b6041a91cd58 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:35.182178 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97605bfc-6ffe-462d-925a-b6041a91cd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:35.182463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97605bfc-6ffe-462d-925a-b6041a91cd58 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:35.183135 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97605bfc-6ffe-462d-925a-b6041a91cd58 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:35.184536 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97605bfc-6ffe-462d-925a-b6041a91cd58 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-97605bfc-6ffe-462d-925a-b6041a91cd58 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:35.194637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:35.195909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:35.196765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:35.196765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:35.196765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:35.196765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:35.196765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:35.196765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:35.196765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:35.196765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:35.196765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:35.196765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:35.196765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:35.196765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:35.196765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:35.196765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:35.200967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 480/956] 10.222.0.100 () {56 vars in 1131 bytes} [Mon Aug 1 15:01:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:35.202065 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97605bfc-6ffe-462d-925a-b6041a91cd58 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:35.228776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d559d-6260-440e-83e5-1f27657c31f9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:35.240488 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:35.272647 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb85e8c-f348-42b5-89ab-9cbc8c0e9151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:35.273812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb85e8c-f348-42b5-89ab-9cbc8c0e9151 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:35.274363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb85e8c-f348-42b5-89ab-9cbc8c0e9151 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:35.274765 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb85e8c-f348-42b5-89ab-9cbc8c0e9151 None admin] PATH_INFO: `/v3/projects/d6e147b6607b481591ceb250f7208c16/users/d0723b88d7cd481cb962072f75d835c7/roles/46f67f665d95448798ec8aef267e78fb` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:35.277155 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb85e8c-f348-42b5-89ab-9cbc8c0e9151 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6e147b6607b481591ceb250f7208c16, user_id=d0723b88d7cd481cb962072f75d835c7, role_id=46f67f665d95448798ec8aef267e78fb)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:35.278820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb85e8c-f348-42b5-89ab-9cbc8c0e9151 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:35.294466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:35.298757 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 477/957] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:01:35 2022] PUT /identity/v3/projects/d6e147b6607b481591ceb250f7208c16/users/d0723b88d7cd481cb962072f75d835c7/roles/46f67f665d95448798ec8aef267e78fb => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:01:35.298757 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323b0d-84a5-4486-bdaf-5e7835d80cbf None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:35.298757 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323b0d-84a5-4486-bdaf-5e7835d80cbf None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:35.298757 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25323b0d-84a5-4486-bdaf-5e7835d80cbf None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:35.299847 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 478/958] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:35.312797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da608f-3d9c-4ec9-b979-4da3d8a4ee5c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:35.313211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da608f-3d9c-4ec9-b979-4da3d8a4ee5c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:35.313473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da608f-3d9c-4ec9-b979-4da3d8a4ee5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:35.324320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97605bfc-6ffe-462d-925a-b6041a91cd58 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:35.339651 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-23da608f-3d9c-4ec9-b979-4da3d8a4ee5c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:35.341756 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-23da608f-3d9c-4ec9-b979-4da3d8a4ee5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:35.372257 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18e969-bb13-4576-ad0e-0adefceeb33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:35.373023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18e969-bb13-4576-ad0e-0adefceeb33b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:35.373302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18e969-bb13-4576-ad0e-0adefceeb33b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:35.373575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18e969-bb13-4576-ad0e-0adefceeb33b None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:35.374337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18e969-bb13-4576-ad0e-0adefceeb33b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:35.374645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18e969-bb13-4576-ad0e-0adefceeb33b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:35.376160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18e969-bb13-4576-ad0e-0adefceeb33b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:35.383045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 481/959] 10.222.0.100 () {56 vars in 1127 bytes} [Mon Aug 1 15:01:35 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:35.389193 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18e969-bb13-4576-ad0e-0adefceeb33b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:35.454542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 479/960] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:35 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:35.480332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da608f-3d9c-4ec9-b979-4da3d8a4ee5c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:35.517256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18e969-bb13-4576-ad0e-0adefceeb33b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:35.564592 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e1605-340f-446f-823c-b8afd408c305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:35.565566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e1605-340f-446f-823c-b8afd408c305 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:35.565995 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e1605-340f-446f-823c-b8afd408c305 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:35.567225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e1605-340f-446f-823c-b8afd408c305 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:35.568620 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e1605-340f-446f-823c-b8afd408c305 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:35.569097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e1605-340f-446f-823c-b8afd408c305 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:35.572377 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e1605-340f-446f-823c-b8afd408c305 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:35.582890 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 482/961] 10.222.0.100 () {56 vars in 1127 bytes} [Mon Aug 1 15:01:35 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:35.592645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e1605-340f-446f-823c-b8afd408c305 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:35.616484 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:35.633550 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:35.638510 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da608f-3d9c-4ec9-b979-4da3d8a4ee5c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:35.689509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c0b5d-65d2-45e4-b57a-42d82071ae6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:35.690142 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c0b5d-65d2-45e4-b57a-42d82071ae6e None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:35.691844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c0b5d-65d2-45e4-b57a-42d82071ae6e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:35.692130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c0b5d-65d2-45e4-b57a-42d82071ae6e None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:35.692834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c0b5d-65d2-45e4-b57a-42d82071ae6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:35.694306 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c0b5d-65d2-45e4-b57a-42d82071ae6e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:35.711407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.sql.core [None req-5c7c0b5d-65d2-45e4-b57a-42d82071ae6e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 15:01:35.711407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 15:01:35.711407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [parameters: {'user_id': 'f5d04222c6904e04ba184a40f14de74e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 15:01:35.711407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80359) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-5c7c0b5d-65d2-45e4-b57a-42d82071ae6e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result.read() Aug 01 15:01:35.719401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:01:35.720340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application yield resource Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application next(self.gen) Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise self.value Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application session.commit() Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 15:01:35.721300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.session.flush() Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application compat.raise_( Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 15:01:35.722245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 15:01:35.723772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application conn.query(q) Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application result.read() Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 15:01:35.724634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f5d04222c6904e04ba184a40f14de74e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:35.726989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 01 15:01:35.728995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 01 15:01:35.731342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 01 15:01:35.731342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 01 15:01:35.731342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 15:01:35.731342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 15:01:35.731342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 15:01:35.731342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:35.731342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 480/962] 10.222.0.100 () {60 vars in 1136 bytes} [Mon Aug 1 15:01:35 2022] POST /identity/v3/users => generated 160 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:35.731342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c0b5d-65d2-45e4-b57a-42d82071ae6e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:35.748525 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e1605-340f-446f-823c-b8afd408c305 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:35.788517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b2223-4f8a-4035-85a2-5102a8e4b7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:35.789575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b2223-4f8a-4035-85a2-5102a8e4b7a0 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:35.790061 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b2223-4f8a-4035-85a2-5102a8e4b7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:35.790561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b2223-4f8a-4035-85a2-5102a8e4b7a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:35.791519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b2223-4f8a-4035-85a2-5102a8e4b7a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:35.793300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b2223-4f8a-4035-85a2-5102a8e4b7a0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:35.822057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 483/963] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:01:35 2022] POST /identity/v3/endpoints => generated 351 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:01:35.861214 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c0b5d-65d2-45e4-b57a-42d82071ae6e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:35.916180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0452d-d3b8-4c95-bbf0-02d126eef184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:35.916821 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0452d-d3b8-4c95-bbf0-02d126eef184 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:35.917109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0452d-d3b8-4c95-bbf0-02d126eef184 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:35.917436 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0452d-d3b8-4c95-bbf0-02d126eef184 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:35.926453 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0452d-d3b8-4c95-bbf0-02d126eef184 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:35.928121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0452d-d3b8-4c95-bbf0-02d126eef184 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-8ff0452d-d3b8-4c95-bbf0-02d126eef184 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:35.942650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:35.943703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:35.944640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:35.944640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:35.944640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:35.944640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:35.944640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:35.944640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:35.944640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:35.944640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:35.944640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 15:01:35.944640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:35.944640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 481/964] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:01:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:35.950698 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b2223-4f8a-4035-85a2-5102a8e4b7a0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:35.970449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bb6cc-c75b-49a9-a9e2-78ac957a9e53 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:35.970817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bb6cc-c75b-49a9-a9e2-78ac957a9e53 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:35.971038 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bb6cc-c75b-49a9-a9e2-78ac957a9e53 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:35.972623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 482/965] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:01:35.978351 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc498dae-f07b-48a3-ad68-1d6f6008cd6e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:35.978712 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc498dae-f07b-48a3-ad68-1d6f6008cd6e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:35.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc498dae-f07b-48a3-ad68-1d6f6008cd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:36.004727 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-fc498dae-f07b-48a3-ad68-1d6f6008cd6e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:36.006773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fc498dae-f07b-48a3-ad68-1d6f6008cd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:36.019593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:36.057494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:36.108763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b2223-4f8a-4035-85a2-5102a8e4b7a0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:36.113300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 483/966] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:35 2022] POST /identity/v3/auth/tokens => generated 1466 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:36.136214 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc498dae-f07b-48a3-ad68-1d6f6008cd6e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:36.158704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acdb4a-f819-4329-ae4c-bc4a23cf23cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:36.160053 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acdb4a-f819-4329-ae4c-bc4a23cf23cd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:36.160857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acdb4a-f819-4329-ae4c-bc4a23cf23cd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:36.161332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acdb4a-f819-4329-ae4c-bc4a23cf23cd None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:36.162262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acdb4a-f819-4329-ae4c-bc4a23cf23cd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:36.162842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acdb4a-f819-4329-ae4c-bc4a23cf23cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:36.165026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acdb4a-f819-4329-ae4c-bc4a23cf23cd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:36.215713 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 484/967] 10.222.0.100 () {56 vars in 1106 bytes} [Mon Aug 1 15:01:35 2022] GET /identity/v3/users?name=cinder => generated 356 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:36.271392 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc498dae-f07b-48a3-ad68-1d6f6008cd6e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:36.313297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167fdd25-fe79-4c08-a534-56c2086b4994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:36.313943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fdd25-fe79-4c08-a534-56c2086b4994 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:36.314226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fdd25-fe79-4c08-a534-56c2086b4994 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:36.314496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167fdd25-fe79-4c08-a534-56c2086b4994 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:36.315267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167fdd25-fe79-4c08-a534-56c2086b4994 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:36.316774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167fdd25-fe79-4c08-a534-56c2086b4994 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:36.334950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 484/968] 10.222.0.100 () {60 vars in 1146 bytes} [Mon Aug 1 15:01:36 2022] POST /identity/v3/services => generated 243 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:01:37.467761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101443-7d38-40bb-8c44-619d4dc7ac9b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:37.468320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101443-7d38-40bb-8c44-619d4dc7ac9b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:37.468320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101443-7d38-40bb-8c44-619d4dc7ac9b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:37.469709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 485/969] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:37.475760 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8be33c-56dc-448d-b0c0-314dba7a3022 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:37.476255 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8be33c-56dc-448d-b0c0-314dba7a3022 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:37.476472 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8be33c-56dc-448d-b0c0-314dba7a3022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:37.492777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:37.499169 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-be8be33c-56dc-448d-b0c0-314dba7a3022 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:37.501039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-be8be33c-56dc-448d-b0c0-314dba7a3022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:37.601212 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 485/970] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:37 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:37.626438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101443-7d38-40bb-8c44-619d4dc7ac9b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:37.738107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101443-7d38-40bb-8c44-619d4dc7ac9b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:37.767873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:37.781554 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5809ee-7f13-47e8-8b14-403e08ebef18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:37.782272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5809ee-7f13-47e8-8b14-403e08ebef18 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:37.782561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5809ee-7f13-47e8-8b14-403e08ebef18 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:37.782873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5809ee-7f13-47e8-8b14-403e08ebef18 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:37.794602 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5809ee-7f13-47e8-8b14-403e08ebef18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-fd5809ee-7f13-47e8-8b14-403e08ebef18 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:37.804467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:37.805476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:37.806361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:37.806361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:37.806361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:37.806361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:37.806361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:37.806361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:37.806361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:37.806361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:37.806361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:37.806361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:37.806361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:01:37.806361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:37.808059 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 486/971] 10.222.0.100 () {56 vars in 1125 bytes} [Mon Aug 1 15:01:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:37.814199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8be33c-56dc-448d-b0c0-314dba7a3022 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:37.928698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8be33c-56dc-448d-b0c0-314dba7a3022 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:37.960261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:37.966337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9876a-563b-492e-a902-d76b0dd6c1c7 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:37.966922 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9876a-563b-492e-a902-d76b0dd6c1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:37.967376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9876a-563b-492e-a902-d76b0dd6c1c7 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:37.967579 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f95d6-28c7-411e-bb63-568b8e9edb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:37.968177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f95d6-28c7-411e-bb63-568b8e9edb36 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:37.968462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f95d6-28c7-411e-bb63-568b8e9edb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:37.968917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f95d6-28c7-411e-bb63-568b8e9edb36 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:37.969034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 487/972] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:01:37.969454 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f95d6-28c7-411e-bb63-568b8e9edb36 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:37.969824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f95d6-28c7-411e-bb63-568b8e9edb36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:37.971188 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f95d6-28c7-411e-bb63-568b8e9edb36 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:37.974638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186262f-cd91-4e81-b331-1c43493933c8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:37.974953 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186262f-cd91-4e81-b331-1c43493933c8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:37.975232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186262f-cd91-4e81-b331-1c43493933c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:37.982440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 486/973] 10.222.0.100 () {56 vars in 1120 bytes} [Mon Aug 1 15:01:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:37.987800 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f95d6-28c7-411e-bb63-568b8e9edb36 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.000080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a186262f-cd91-4e81-b331-1c43493933c8 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:38.001211 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a186262f-cd91-4e81-b331-1c43493933c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:38.090590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 488/974] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:37 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:38.113488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186262f-cd91-4e81-b331-1c43493933c8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.123175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f95d6-28c7-411e-bb63-568b8e9edb36 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:38.173480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b141f5-7f4a-4a3f-b9a1-75f66c5deb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:38.174137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b141f5-7f4a-4a3f-b9a1-75f66c5deb7d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.174420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b141f5-7f4a-4a3f-b9a1-75f66c5deb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.174721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b141f5-7f4a-4a3f-b9a1-75f66c5deb7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.186764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b141f5-7f4a-4a3f-b9a1-75f66c5deb7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:38.188527 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b141f5-7f4a-4a3f-b9a1-75f66c5deb7d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-84b141f5-7f4a-4a3f-b9a1-75f66c5deb7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:38.204700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:38.205666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:38.206731 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:38.206731 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:38.206731 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:38.206731 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:38.206731 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 487/975] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:38.213029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b141f5-7f4a-4a3f-b9a1-75f66c5deb7d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.233150 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186262f-cd91-4e81-b331-1c43493933c8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:38.252158 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:38.272123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:38.273210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c523f-4822-4c52-aa93-9ecd5027aa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:38.273968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c523f-4822-4c52-aa93-9ecd5027aa96 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.274203 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c523f-4822-4c52-aa93-9ecd5027aa96 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.274865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c523f-4822-4c52-aa93-9ecd5027aa96 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.275577 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c523f-4822-4c52-aa93-9ecd5027aa96 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:38.277274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c523f-4822-4c52-aa93-9ecd5027aa96 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5e5c523f-4822-4c52-aa93-9ecd5027aa96 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:38.287334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:38.288411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:38.289870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 489/976] 10.222.0.100 () {56 vars in 1131 bytes} [Mon Aug 1 15:01:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:38.294429 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c523f-4822-4c52-aa93-9ecd5027aa96 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.348720 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b141f5-7f4a-4a3f-b9a1-75f66c5deb7d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:38.388249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34e8aa-5322-4622-acaa-5557e8e2157e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:38.388883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34e8aa-5322-4622-acaa-5557e8e2157e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.389165 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34e8aa-5322-4622-acaa-5557e8e2157e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.389440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34e8aa-5322-4622-acaa-5557e8e2157e None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.390136 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34e8aa-5322-4622-acaa-5557e8e2157e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:38.390441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34e8aa-5322-4622-acaa-5557e8e2157e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:38.391876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34e8aa-5322-4622-acaa-5557e8e2157e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:38.407989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 488/977] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:38.411470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c523f-4822-4c52-aa93-9ecd5027aa96 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:38.413384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34e8aa-5322-4622-acaa-5557e8e2157e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.447623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3c141-dbb1-43d8-aa10-c410e5210c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:38.448339 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3c141-dbb1-43d8-aa10-c410e5210c5b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.448627 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3c141-dbb1-43d8-aa10-c410e5210c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.448906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3c141-dbb1-43d8-aa10-c410e5210c5b None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.449579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3c141-dbb1-43d8-aa10-c410e5210c5b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:38.449958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3c141-dbb1-43d8-aa10-c410e5210c5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:38.451351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3c141-dbb1-43d8-aa10-c410e5210c5b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:38.456876 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 490/978] 10.222.0.100 () {56 vars in 1127 bytes} [Mon Aug 1 15:01:38 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:38.458632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8136aed-581a-4813-819c-0207000a3c5a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.458949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8136aed-581a-4813-819c-0207000a3c5a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.459226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8136aed-581a-4813-819c-0207000a3c5a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.460579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 491/979] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:38.465165 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8136aed-581a-4813-819c-0207000a3c5a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.513871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:38.522543 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34e8aa-5322-4622-acaa-5557e8e2157e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:38.530046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:38.560069 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9a36a-e0b5-48c9-a6f2-f5074a32e0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:38.560706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9a36a-e0b5-48c9-a6f2-f5074a32e0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.560988 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9a36a-e0b5-48c9-a6f2-f5074a32e0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.561264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9a36a-e0b5-48c9-a6f2-f5074a32e0a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.569290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9a36a-e0b5-48c9-a6f2-f5074a32e0a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:38.570937 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c9a36a-e0b5-48c9-a6f2-f5074a32e0a1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:38.574031 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8136aed-581a-4813-819c-0207000a3c5a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-51c9a36a-e0b5-48c9-a6f2-f5074a32e0a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:38.582528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:38.583525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:38.584493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:38.584493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:38.584493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:38.584493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:38.584493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:38.584493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:38.584493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:38.584493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:38.584493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 15:01:38.584493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:38.584493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 489/980] 10.222.0.100 () {56 vars in 1122 bytes} [Mon Aug 1 15:01:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:38.585128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2827dd-266a-4b33-a9bf-4f975f6e8486 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.585128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2827dd-266a-4b33-a9bf-4f975f6e8486 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.585747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2827dd-266a-4b33-a9bf-4f975f6e8486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.604579 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-bd2827dd-266a-4b33-a9bf-4f975f6e8486 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:38.605794 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bd2827dd-266a-4b33-a9bf-4f975f6e8486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:38.614372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781eadb-23d8-4804-ac0f-c206573dd3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:38.615023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781eadb-23d8-4804-ac0f-c206573dd3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.615307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781eadb-23d8-4804-ac0f-c206573dd3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.615581 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781eadb-23d8-4804-ac0f-c206573dd3a8 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.616258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781eadb-23d8-4804-ac0f-c206573dd3a8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:38.616560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781eadb-23d8-4804-ac0f-c206573dd3a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:38.617938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781eadb-23d8-4804-ac0f-c206573dd3a8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:38.622755 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 492/981] 10.222.0.100 () {56 vars in 1127 bytes} [Mon Aug 1 15:01:38 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:38.623612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781eadb-23d8-4804-ac0f-c206573dd3a8 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.674820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 490/982] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:38 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:38.676467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2827dd-266a-4b33-a9bf-4f975f6e8486 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.743896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781eadb-23d8-4804-ac0f-c206573dd3a8 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:38.756117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:38.781912 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:38.785862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ee94a-00e3-4631-8155-b57fd5d45464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:38.786554 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ee94a-00e3-4631-8155-b57fd5d45464 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.786811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ee94a-00e3-4631-8155-b57fd5d45464 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.787080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ee94a-00e3-4631-8155-b57fd5d45464 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.787883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ee94a-00e3-4631-8155-b57fd5d45464 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:38.788197 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ee94a-00e3-4631-8155-b57fd5d45464 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:38.788612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2827dd-266a-4b33-a9bf-4f975f6e8486 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:38.789649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ee94a-00e3-4631-8155-b57fd5d45464 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:38.808394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 493/983] 10.222.0.100 () {56 vars in 1154 bytes} [Mon Aug 1 15:01:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:38.809248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ee94a-00e3-4631-8155-b57fd5d45464 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.830005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520c626-6e97-452d-bee2-1b0e20232fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:38.830632 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520c626-6e97-452d-bee2-1b0e20232fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.830938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520c626-6e97-452d-bee2-1b0e20232fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.831210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520c626-6e97-452d-bee2-1b0e20232fe0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.831978 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520c626-6e97-452d-bee2-1b0e20232fe0 None admin] RBAC: Adding query filter params (interface=admin, service_id=3557d4234b5c458696e6edbf790b3e98, region_id=RegionOne) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:38.832317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520c626-6e97-452d-bee2-1b0e20232fe0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:38.833695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520c626-6e97-452d-bee2-1b0e20232fe0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:38.842649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 491/984] 10.222.0.100 () {56 vars in 1254 bytes} [Mon Aug 1 15:01:38 2022] GET /identity/v3/endpoints?service_id=3557d4234b5c458696e6edbf790b3e98&interface=admin®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:38.843508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520c626-6e97-452d-bee2-1b0e20232fe0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.928399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ee94a-00e3-4631-8155-b57fd5d45464 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:38.953281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520c626-6e97-452d-bee2-1b0e20232fe0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:38.963923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea69179-cef5-4651-92eb-cd3559869186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:38.964576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea69179-cef5-4651-92eb-cd3559869186 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.964860 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea69179-cef5-4651-92eb-cd3559869186 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.965146 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea69179-cef5-4651-92eb-cd3559869186 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:38.965785 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea69179-cef5-4651-92eb-cd3559869186 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:38.967214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea69179-cef5-4651-92eb-cd3559869186 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-eea69179-cef5-4651-92eb-cd3559869186 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:38.976906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:38.978069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:38.979070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 494/985] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:38.979899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea69179-cef5-4651-92eb-cd3559869186 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:38.993497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6ceb9-b748-4102-bc35-e5815830936c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:38.994121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6ceb9-b748-4102-bc35-e5815830936c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:38.994405 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6ceb9-b748-4102-bc35-e5815830936c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:38.994703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6ceb9-b748-4102-bc35-e5815830936c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:39.015217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:39.023957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:39.066974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6ceb9-b748-4102-bc35-e5815830936c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:39.068363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6ceb9-b748-4102-bc35-e5815830936c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:39.089502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea69179-cef5-4651-92eb-cd3559869186 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:39.125523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c151e-3ffc-48fb-95ba-c2b1ac5d0f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:39.126143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c151e-3ffc-48fb-95ba-c2b1ac5d0f58 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:39.126426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c151e-3ffc-48fb-95ba-c2b1ac5d0f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:39.126727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c151e-3ffc-48fb-95ba-c2b1ac5d0f58 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:39.127460 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c151e-3ffc-48fb-95ba-c2b1ac5d0f58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:39.128834 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c151e-3ffc-48fb-95ba-c2b1ac5d0f58 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:39.135544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 495/986] 10.222.0.100 () {56 vars in 1092 bytes} [Mon Aug 1 15:01:38 2022] GET /identity/v3/services => generated 1735 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:01:39.136323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c151e-3ffc-48fb-95ba-c2b1ac5d0f58 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:39.192263 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 492/987] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:38 2022] GET /identity/v3/auth/tokens => generated 1562 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:01:39.198694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6ceb9-b748-4102-bc35-e5815830936c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:39.249702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:39.259279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c151e-3ffc-48fb-95ba-c2b1ac5d0f58 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:39.279847 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:39.306987 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f69a89-9e80-4523-8bcb-bf7c0c1230d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:39.308011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f69a89-9e80-4523-8bcb-bf7c0c1230d5 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:39.308011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f69a89-9e80-4523-8bcb-bf7c0c1230d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:39.308211 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f69a89-9e80-4523-8bcb-bf7c0c1230d5 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:39.308984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f69a89-9e80-4523-8bcb-bf7c0c1230d5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:39.309247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f69a89-9e80-4523-8bcb-bf7c0c1230d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:39.310677 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f69a89-9e80-4523-8bcb-bf7c0c1230d5 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:39.318773 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 496/988] 10.222.0.100 () {56 vars in 1117 bytes} [Mon Aug 1 15:01:39 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:39.323896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f69a89-9e80-4523-8bcb-bf7c0c1230d5 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:39.341934 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6ceb9-b748-4102-bc35-e5815830936c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:39.384900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b362500-2814-46c7-a27d-a1f5f19ffa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:39.385661 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b362500-2814-46c7-a27d-a1f5f19ffa75 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:39.385948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b362500-2814-46c7-a27d-a1f5f19ffa75 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:39.386225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b362500-2814-46c7-a27d-a1f5f19ffa75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:39.398765 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b362500-2814-46c7-a27d-a1f5f19ffa75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:39.400266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b362500-2814-46c7-a27d-a1f5f19ffa75 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-0b362500-2814-46c7-a27d-a1f5f19ffa75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:39.420608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:39.421638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:39.422565 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:39.422565 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:39.422565 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:39.422565 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:39.426488 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 493/989] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:39.427493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b362500-2814-46c7-a27d-a1f5f19ffa75 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:39.438455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f69a89-9e80-4523-8bcb-bf7c0c1230d5 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:39.478073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e18f29-bc43-4593-a093-585b574d43b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:39.478844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e18f29-bc43-4593-a093-585b574d43b4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:39.479135 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e18f29-bc43-4593-a093-585b574d43b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:39.479410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e18f29-bc43-4593-a093-585b574d43b4 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:39.480101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e18f29-bc43-4593-a093-585b574d43b4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:39.480411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e18f29-bc43-4593-a093-585b574d43b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:39.481795 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e18f29-bc43-4593-a093-585b574d43b4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:39.487122 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 497/990] 10.222.0.100 () {56 vars in 1117 bytes} [Mon Aug 1 15:01:39 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:39.493545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e18f29-bc43-4593-a093-585b574d43b4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:39.524142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:39.572008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b362500-2814-46c7-a27d-a1f5f19ffa75 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:39.588164 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:39.616657 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e18f29-bc43-4593-a093-585b574d43b4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:39.618495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d47e9-2bae-49b9-8b51-16b26f034e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:39.619927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d47e9-2bae-49b9-8b51-16b26f034e1e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:39.620236 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d47e9-2bae-49b9-8b51-16b26f034e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:39.620585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d47e9-2bae-49b9-8b51-16b26f034e1e None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:39.621387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d47e9-2bae-49b9-8b51-16b26f034e1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:39.621772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d47e9-2bae-49b9-8b51-16b26f034e1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:39.623173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76d47e9-2bae-49b9-8b51-16b26f034e1e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:39.638985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 494/991] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:39.645969 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d47e9-2bae-49b9-8b51-16b26f034e1e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:39.658001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c84bf-891f-493a-94f6-6a6ee06d7eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:39.658620 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c84bf-891f-493a-94f6-6a6ee06d7eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:39.658907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c84bf-891f-493a-94f6-6a6ee06d7eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:39.659187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c84bf-891f-493a-94f6-6a6ee06d7eb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:39.659951 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c84bf-891f-493a-94f6-6a6ee06d7eb4 None admin] RBAC: Adding query filter params (interface=public, service_id=d2925334161d4adeb9e595bc0793306e, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:39.660316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c84bf-891f-493a-94f6-6a6ee06d7eb4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:39.661740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c84bf-891f-493a-94f6-6a6ee06d7eb4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:39.672315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 498/992] 10.222.0.100 () {56 vars in 1256 bytes} [Mon Aug 1 15:01:39 2022] GET /identity/v3/endpoints?service_id=d2925334161d4adeb9e595bc0793306e&interface=public®ion_id=RegionOne => generated 195 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:39.673181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c84bf-891f-493a-94f6-6a6ee06d7eb4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:39.766369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76d47e9-2bae-49b9-8b51-16b26f034e1e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:39.783175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:39.797802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c84bf-891f-493a-94f6-6a6ee06d7eb4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:39.805230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c1454-95f8-4173-84f8-2dc7bc9754c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:39.805889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c1454-95f8-4173-84f8-2dc7bc9754c1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:39.806164 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c1454-95f8-4173-84f8-2dc7bc9754c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:39.806445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c1454-95f8-4173-84f8-2dc7bc9754c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:39.813966 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c1454-95f8-4173-84f8-2dc7bc9754c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:39.815467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c1454-95f8-4173-84f8-2dc7bc9754c1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-761c1454-95f8-4173-84f8-2dc7bc9754c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:39.828157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:39.829134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:39.830032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:39.830032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:39.830032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:39.830032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:39.830032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:39.830032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:39.830032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:39.830032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:39.830032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 495/993] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:39.835954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c1454-95f8-4173-84f8-2dc7bc9754c1 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:39.842236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d268b-a546-4ef2-ba28-ea157feeff65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:39.842895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d268b-a546-4ef2-ba28-ea157feeff65 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:39.843183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d268b-a546-4ef2-ba28-ea157feeff65 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:39.843457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d268b-a546-4ef2-ba28-ea157feeff65 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:39.844116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d268b-a546-4ef2-ba28-ea157feeff65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:39.845500 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d268b-a546-4ef2-ba28-ea157feeff65 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:39.862698 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 499/994] 10.222.0.100 () {56 vars in 1092 bytes} [Mon Aug 1 15:01:39 2022] GET /identity/v3/services => generated 1735 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:01:39.884199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:39.968663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c1454-95f8-4173-84f8-2dc7bc9754c1 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:40.009118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971edb7-5d77-4acf-becb-446627dae114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:40.009758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971edb7-5d77-4acf-becb-446627dae114 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:40.010039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971edb7-5d77-4acf-becb-446627dae114 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:40.010313 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971edb7-5d77-4acf-becb-446627dae114 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:40.011124 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971edb7-5d77-4acf-becb-446627dae114 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:40.011403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971edb7-5d77-4acf-becb-446627dae114 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:40.012852 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971edb7-5d77-4acf-becb-446627dae114 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:40.026607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 496/995] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:40.033009 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d268b-a546-4ef2-ba28-ea157feeff65 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:40.153982 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d268b-a546-4ef2-ba28-ea157feeff65 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:40.193929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35392e87-84f2-41c1-8726-fa2d6608d1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:40.194567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35392e87-84f2-41c1-8726-fa2d6608d1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:40.194878 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35392e87-84f2-41c1-8726-fa2d6608d1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:40.195156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35392e87-84f2-41c1-8726-fa2d6608d1f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:40.195957 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35392e87-84f2-41c1-8726-fa2d6608d1f1 None admin] RBAC: Adding query filter params (role.id=46f67f665d95448798ec8aef267e78fb, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=d0723b88d7cd481cb962072f75d835c7) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:40.196269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35392e87-84f2-41c1-8726-fa2d6608d1f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:40.197715 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35392e87-84f2-41c1-8726-fa2d6608d1f1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:40.204682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 500/996] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:40 2022] GET /identity/v3/role_assignments?user.id=d0723b88d7cd481cb962072f75d835c7&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=46f67f665d95448798ec8aef267e78fb => generated 610 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:41.204429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc154747-348c-453e-bbf6-99d0d9c834a7 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:41.205067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc154747-348c-453e-bbf6-99d0d9c834a7 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:41.205067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc154747-348c-453e-bbf6-99d0d9c834a7 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:41.206363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 497/997] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:41.211674 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fce4ae-347e-4353-ba4e-d767d08fa81b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:41.212376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fce4ae-347e-4353-ba4e-d767d08fa81b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:41.212487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fce4ae-347e-4353-ba4e-d767d08fa81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:41.231280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b0fce4ae-347e-4353-ba4e-d767d08fa81b None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:41.232472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b0fce4ae-347e-4353-ba4e-d767d08fa81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:41.244965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:41.316996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 501/998] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:41 2022] POST /identity/v3/auth/tokens => generated 1562 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:41.340187 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc154747-348c-453e-bbf6-99d0d9c834a7 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:41.449285 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:41.461628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc154747-348c-453e-bbf6-99d0d9c834a7 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:41.504104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008af218-b70f-4547-bd9e-50dad1667daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:41.504752 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008af218-b70f-4547-bd9e-50dad1667daf None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:41.505031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008af218-b70f-4547-bd9e-50dad1667daf None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:41.505304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008af218-b70f-4547-bd9e-50dad1667daf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:41.505957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008af218-b70f-4547-bd9e-50dad1667daf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:41.507360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008af218-b70f-4547-bd9e-50dad1667daf None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-008af218-b70f-4547-bd9e-50dad1667daf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:41.516860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:41.517914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:41.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 498/999] 10.222.0.100 () {56 vars in 1131 bytes} [Mon Aug 1 15:01:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:41.524721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fce4ae-347e-4353-ba4e-d767d08fa81b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:41.645372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fce4ae-347e-4353-ba4e-d767d08fa81b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:41.682594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca7b15-f155-4b19-a270-b7053454cbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:41.683526 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca7b15-f155-4b19-a270-b7053454cbe2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:41.683965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca7b15-f155-4b19-a270-b7053454cbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:41.684535 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca7b15-f155-4b19-a270-b7053454cbe2 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:41.685465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca7b15-f155-4b19-a270-b7053454cbe2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:41.685938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca7b15-f155-4b19-a270-b7053454cbe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:41.687519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca7b15-f155-4b19-a270-b7053454cbe2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:41.698726 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 502/1000] 10.222.0.100 () {56 vars in 1127 bytes} [Mon Aug 1 15:01:41 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:41.699600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008af218-b70f-4547-bd9e-50dad1667daf None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:41.818475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008af218-b70f-4547-bd9e-50dad1667daf None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:41.854999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e35dc-b6b9-4bfc-b147-9b9d1b7f40bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:41.855885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e35dc-b6b9-4bfc-b147-9b9d1b7f40bd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:41.856318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e35dc-b6b9-4bfc-b147-9b9d1b7f40bd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:41.856746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e35dc-b6b9-4bfc-b147-9b9d1b7f40bd None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:41.857670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e35dc-b6b9-4bfc-b147-9b9d1b7f40bd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:41.858145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e35dc-b6b9-4bfc-b147-9b9d1b7f40bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:41.859914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e35dc-b6b9-4bfc-b147-9b9d1b7f40bd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:41.870962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 499/1001] 10.222.0.100 () {56 vars in 1127 bytes} [Mon Aug 1 15:01:41 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:41.872629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca7b15-f155-4b19-a270-b7053454cbe2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:41.886287 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:41.959396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f0496-d2b8-4801-ad68-20b8b37c4a46 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:41.959689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f0496-d2b8-4801-ad68-20b8b37c4a46 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:41.959969 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f0496-d2b8-4801-ad68-20b8b37c4a46 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:41.961386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 500/1002] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:41.968486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0a74f-c985-4ef7-9a73-da49ee1810ea None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:41.968798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0a74f-c985-4ef7-9a73-da49ee1810ea None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:41.969207 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0a74f-c985-4ef7-9a73-da49ee1810ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:41.988953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-fbb0a74f-c985-4ef7-9a73-da49ee1810ea None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:41.990178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fbb0a74f-c985-4ef7-9a73-da49ee1810ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:41.996855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca7b15-f155-4b19-a270-b7053454cbe2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:42.006085 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:42.039384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6a037-2bd5-4a4a-978c-bba3c8677beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:42.040014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6a037-2bd5-4a4a-978c-bba3c8677beb None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:42.040335 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6a037-2bd5-4a4a-978c-bba3c8677beb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:42.040575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6a037-2bd5-4a4a-978c-bba3c8677beb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:42.041324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6a037-2bd5-4a4a-978c-bba3c8677beb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:42.042751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6a037-2bd5-4a4a-978c-bba3c8677beb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:42.062386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 503/1003] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:01:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:01:42.081710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 501/1004] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:41 2022] POST /identity/v3/auth/tokens => generated 1710 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:42.105919 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6a037-2bd5-4a4a-978c-bba3c8677beb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:42.195065 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:42.236771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6a037-2bd5-4a4a-978c-bba3c8677beb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:42.275817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f4f84f-6dcd-480c-a4ef-baebdabaf287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:42.276479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4f84f-6dcd-480c-a4ef-baebdabaf287 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:42.276762 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4f84f-6dcd-480c-a4ef-baebdabaf287 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:42.277041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f4f84f-6dcd-480c-a4ef-baebdabaf287 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:42.277679 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4f84f-6dcd-480c-a4ef-baebdabaf287 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:42.279166 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f4f84f-6dcd-480c-a4ef-baebdabaf287 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-52f4f84f-6dcd-480c-a4ef-baebdabaf287 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:42.289289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:42.290397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:42.291370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 504/1005] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:42.296530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0a74f-c985-4ef7-9a73-da49ee1810ea None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:42.320235 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb1e1f-c9f5-4c35-a01c-bef20e0d43ae None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:42.320561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb1e1f-c9f5-4c35-a01c-bef20e0d43ae None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:42.320796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb1e1f-c9f5-4c35-a01c-bef20e0d43ae None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:42.322418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 505/1006] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:01:42.327721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00768da-c487-46c3-9dc3-71e34dde667f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:42.328014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00768da-c487-46c3-9dc3-71e34dde667f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:42.328291 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00768da-c487-46c3-9dc3-71e34dde667f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:42.353100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b00768da-c487-46c3-9dc3-71e34dde667f None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:42.354333 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b00768da-c487-46c3-9dc3-71e34dde667f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:42.422884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0a74f-c985-4ef7-9a73-da49ee1810ea None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:42.430161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 506/1007] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:42 2022] POST /identity/v3/auth/tokens => generated 1710 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:42.452174 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00768da-c487-46c3-9dc3-71e34dde667f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:42.460509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af289a-21a2-48cc-b690-a0280c6722fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:42.461123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af289a-21a2-48cc-b690-a0280c6722fc None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:42.461406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af289a-21a2-48cc-b690-a0280c6722fc None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:42.461677 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89af289a-21a2-48cc-b690-a0280c6722fc None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:42.462626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af289a-21a2-48cc-b690-a0280c6722fc None admin] RBAC: Adding query filter params (name=compute) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:42.462958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af289a-21a2-48cc-b690-a0280c6722fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:42.464331 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89af289a-21a2-48cc-b690-a0280c6722fc None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:42.474468 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 502/1008] 10.222.0.100 () {56 vars in 1117 bytes} [Mon Aug 1 15:01:42 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:42.475677 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af289a-21a2-48cc-b690-a0280c6722fc None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:42.491209 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:42.562833 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:42.605481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89af289a-21a2-48cc-b690-a0280c6722fc None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:42.609229 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00768da-c487-46c3-9dc3-71e34dde667f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:42.649735 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890ec63-b214-458e-895a-b995c539fcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:42.650355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890ec63-b214-458e-895a-b995c539fcdb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:42.650633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890ec63-b214-458e-895a-b995c539fcdb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:42.650939 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9890ec63-b214-458e-895a-b995c539fcdb None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:42.651629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890ec63-b214-458e-895a-b995c539fcdb None admin] RBAC: Adding query filter params (type=compute) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:42.651941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890ec63-b214-458e-895a-b995c539fcdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:42.653453 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9890ec63-b214-458e-895a-b995c539fcdb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:42.655300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4f123-f23b-4bdd-8df7-87c86e872fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:42.656039 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f123-f23b-4bdd-8df7-87c86e872fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:42.656394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f123-f23b-4bdd-8df7-87c86e872fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:42.656680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f123-f23b-4bdd-8df7-87c86e872fc4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:42.657325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4f123-f23b-4bdd-8df7-87c86e872fc4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:42.658749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4f123-f23b-4bdd-8df7-87c86e872fc4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:42.664277 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 503/1009] 10.222.0.100 () {56 vars in 1117 bytes} [Mon Aug 1 15:01:42 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:42.665180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890ec63-b214-458e-895a-b995c539fcdb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-01f4f123-f23b-4bdd-8df7-87c86e872fc4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:42.672609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:42.674042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:42.674937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:42.674937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:42.674937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:42.674937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:42.674937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:42.674937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:42.674937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:42.674937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:42.674937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:42.674937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:42.674937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:42.674937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:42.674937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:01:42.674937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:42.678731 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 507/1010] 10.222.0.100 () {56 vars in 1110 bytes} [Mon Aug 1 15:01:42 2022] GET /identity/v3/services/image => generated 86 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:42.679909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4f123-f23b-4bdd-8df7-87c86e872fc4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:42.749095 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:42.780682 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9890ec63-b214-458e-895a-b995c539fcdb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:42.792173 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4f123-f23b-4bdd-8df7-87c86e872fc4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:42.821173 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:42.824207 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752b269-9ad2-4b6c-a2fb-ab1d624978c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:42.825191 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752b269-9ad2-4b6c-a2fb-ab1d624978c4 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:42.825615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752b269-9ad2-4b6c-a2fb-ab1d624978c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:42.826041 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752b269-9ad2-4b6c-a2fb-ab1d624978c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:42.827009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752b269-9ad2-4b6c-a2fb-ab1d624978c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:42.828865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a752b269-9ad2-4b6c-a2fb-ab1d624978c4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:42.829654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bba64-eb00-42e1-a6cb-807c67d105d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:42.830276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bba64-eb00-42e1-a6cb-807c67d105d2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:42.830570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bba64-eb00-42e1-a6cb-807c67d105d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:42.830992 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57bba64-eb00-42e1-a6cb-807c67d105d2 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:42.831724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bba64-eb00-42e1-a6cb-807c67d105d2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:42.832806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bba64-eb00-42e1-a6cb-807c67d105d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:42.834149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57bba64-eb00-42e1-a6cb-807c67d105d2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:42.839965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 508/1011] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:42 2022] GET /identity/v3/services?name=image => generated 123 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:42.846321 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bba64-eb00-42e1-a6cb-807c67d105d2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:42.868231 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 504/1012] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:01:42 2022] POST /identity/v3/endpoints => generated 336 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:01:42.974511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57bba64-eb00-42e1-a6cb-807c67d105d2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:43.028534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1280d-2e34-483f-841b-29e8b216a911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:43.029160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1280d-2e34-483f-841b-29e8b216a911 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:43.029439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1280d-2e34-483f-841b-29e8b216a911 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:43.029706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1280d-2e34-483f-841b-29e8b216a911 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:43.030386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1280d-2e34-483f-841b-29e8b216a911 None admin] RBAC: Adding query filter params (type=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:43.030717 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1280d-2e34-483f-841b-29e8b216a911 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:43.032090 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1280d-2e34-483f-841b-29e8b216a911 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:43.037317 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 509/1013] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:42 2022] GET /identity/v3/services?type=image => generated 123 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:44.151476 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ba699-0bf6-4a69-bd66-faca89b36bc5 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:44.152026 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ba699-0bf6-4a69-bd66-faca89b36bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:44.152026 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ba699-0bf6-4a69-bd66-faca89b36bc5 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:44.153448 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 505/1014] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:44.159564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb09fa-f201-43ea-9e85-e6badb98c195 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:44.160636 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb09fa-f201-43ea-9e85-e6badb98c195 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:44.160636 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb09fa-f201-43ea-9e85-e6badb98c195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:44.180055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-68bb09fa-f201-43ea-9e85-e6badb98c195 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:44.181238 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-68bb09fa-f201-43ea-9e85-e6badb98c195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:44.231181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:44.254523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 510/1015] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:44 2022] POST /identity/v3/auth/tokens => generated 1867 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:44.277345 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ba699-0bf6-4a69-bd66-faca89b36bc5 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:44.383549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ba699-0bf6-4a69-bd66-faca89b36bc5 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:44.417197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:44.425490 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d32f8b-cfcf-447e-b8c0-214bfab8c787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:44.426792 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d32f8b-cfcf-447e-b8c0-214bfab8c787 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:44.427071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d32f8b-cfcf-447e-b8c0-214bfab8c787 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:44.427339 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d32f8b-cfcf-447e-b8c0-214bfab8c787 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:44.428656 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d32f8b-cfcf-447e-b8c0-214bfab8c787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:44.430490 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d32f8b-cfcf-447e-b8c0-214bfab8c787 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:44.436838 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 506/1016] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:44 2022] GET /identity/v3/domains/default => generated 200 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:44.438152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb09fa-f201-43ea-9e85-e6badb98c195 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:44.556470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb09fa-f201-43ea-9e85-e6badb98c195 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:44.604261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db44fc-ab78-4587-8fc7-c12662877093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:44.604908 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db44fc-ab78-4587-8fc7-c12662877093 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:44.605193 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db44fc-ab78-4587-8fc7-c12662877093 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:44.605471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db44fc-ab78-4587-8fc7-c12662877093 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:44.606190 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db44fc-ab78-4587-8fc7-c12662877093 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:44.607675 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db44fc-ab78-4587-8fc7-c12662877093 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:44.635828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 511/1017] 10.222.0.100 () {60 vars in 1146 bytes} [Mon Aug 1 15:01:44 2022] POST /identity/v3/projects => generated 312 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:01:45.021392 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26883b-1c26-44e4-b6f1-86fa72b8e468 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:45.021688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26883b-1c26-44e4-b6f1-86fa72b8e468 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:45.021962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26883b-1c26-44e4-b6f1-86fa72b8e468 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:45.023624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 507/1018] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:45.029249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7d5b8-a986-452a-94cd-910086f81655 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:45.029630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7d5b8-a986-452a-94cd-910086f81655 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:45.029867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7d5b8-a986-452a-94cd-910086f81655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:45.047983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-54e7d5b8-a986-452a-94cd-910086f81655 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:45.049206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-54e7d5b8-a986-452a-94cd-910086f81655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:45.079271 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:45.084701 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a091a2c-0c41-4f2b-a9e3-b84918f331ef None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:45.084958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a091a2c-0c41-4f2b-a9e3-b84918f331ef None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:45.085198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a091a2c-0c41-4f2b-a9e3-b84918f331ef None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:45.086714 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 508/1019] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:01:45.092167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a383d7-cb97-4efa-ac3b-968979aa4764 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:45.092457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a383d7-cb97-4efa-ac3b-968979aa4764 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:45.093512 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a383d7-cb97-4efa-ac3b-968979aa4764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:45.112126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-80a383d7-cb97-4efa-ac3b-968979aa4764 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:45.113401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-80a383d7-cb97-4efa-ac3b-968979aa4764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:45.133216 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 512/1020] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:45 2022] POST /identity/v3/auth/tokens => generated 1867 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:45.157532 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7d5b8-a986-452a-94cd-910086f81655 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:45.199071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 509/1021] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:45 2022] POST /identity/v3/auth/tokens => generated 1867 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:45.222444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a383d7-cb97-4efa-ac3b-968979aa4764 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:45.280370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7d5b8-a986-452a-94cd-910086f81655 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:45.322533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65831a-2ef9-47d0-9872-1ce8b15d0a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:45.323240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65831a-2ef9-47d0-9872-1ce8b15d0a36 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:45.323526 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65831a-2ef9-47d0-9872-1ce8b15d0a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:45.323804 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65831a-2ef9-47d0-9872-1ce8b15d0a36 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:45.337824 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc65831a-2ef9-47d0-9872-1ce8b15d0a36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-bc65831a-2ef9-47d0-9872-1ce8b15d0a36 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:45.342842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:45.343831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:45.344693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:45.344693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:45.344693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:45.344693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:45.344693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:45.344693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:45.344693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:45.344693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:45.344693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:45.344693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:45.344693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:01:45.344693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:45.345386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a383d7-cb97-4efa-ac3b-968979aa4764 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:45.345467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 513/1022] 10.222.0.100 () {56 vars in 1125 bytes} [Mon Aug 1 15:01:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:45.350006 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65831a-2ef9-47d0-9872-1ce8b15d0a36 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:45.381436 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:45.387490 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:45.398254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c2031-c1c7-48f2-a5f1-bee7f5b751e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:45.399043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2031-c1c7-48f2-a5f1-bee7f5b751e1 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:45.399327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2031-c1c7-48f2-a5f1-bee7f5b751e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:45.399607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c2031-c1c7-48f2-a5f1-bee7f5b751e1 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:45.400319 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c2031-c1c7-48f2-a5f1-bee7f5b751e1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:45.401823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c2031-c1c7-48f2-a5f1-bee7f5b751e1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:45.426427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 510/1023] 10.222.0.100 () {60 vars in 1146 bytes} [Mon Aug 1 15:01:45 2022] POST /identity/v3/services => generated 243 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:01:45.482723 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65831a-2ef9-47d0-9872-1ce8b15d0a36 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:45.532310 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f6362-43e2-4100-9b04-b1a937f1d7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:45.532853 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f6362-43e2-4100-9b04-b1a937f1d7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:45.533132 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f6362-43e2-4100-9b04-b1a937f1d7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:45.533461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f6362-43e2-4100-9b04-b1a937f1d7c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:45.534195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f6362-43e2-4100-9b04-b1a937f1d7c1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:45.534499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f6362-43e2-4100-9b04-b1a937f1d7c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:45.535980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f6362-43e2-4100-9b04-b1a937f1d7c1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:45.553303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 514/1024] 10.222.0.100 () {56 vars in 1120 bytes} [Mon Aug 1 15:01:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:45.559681 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c2031-c1c7-48f2-a5f1-bee7f5b751e1 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:45.681527 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c2031-c1c7-48f2-a5f1-bee7f5b751e1 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:45.716464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d2535-9de8-4279-837b-62efa606458d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:45.717205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d2535-9de8-4279-837b-62efa606458d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:45.717555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d2535-9de8-4279-837b-62efa606458d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:45.717899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d2535-9de8-4279-837b-62efa606458d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:45.732745 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d2535-9de8-4279-837b-62efa606458d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:45.734422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d2535-9de8-4279-837b-62efa606458d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-cc5d2535-9de8-4279-837b-62efa606458d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:45.753027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:45.754284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:45.755505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:45.755505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:45.755505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:45.755505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:45.755505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 511/1025] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:45.760935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f6362-43e2-4100-9b04-b1a937f1d7c1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:45.857622 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:45.877938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f6362-43e2-4100-9b04-b1a937f1d7c1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:45.928331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a529f-da3d-4da6-a99d-345c1afbbc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:45.929054 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a529f-da3d-4da6-a99d-345c1afbbc9f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:45.929410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a529f-da3d-4da6-a99d-345c1afbbc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:45.929757 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a529f-da3d-4da6-a99d-345c1afbbc9f None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:45.930528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a529f-da3d-4da6-a99d-345c1afbbc9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:45.930934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a529f-da3d-4da6-a99d-345c1afbbc9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:45.932375 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a529f-da3d-4da6-a99d-345c1afbbc9f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:45.951612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 515/1026] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:45.953650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d2535-9de8-4279-837b-62efa606458d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:46.011278 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:46.076561 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d2535-9de8-4279-837b-62efa606458d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:46.118459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db82a8-7ede-439c-8e74-08fa7e3aba92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:46.119067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db82a8-7ede-439c-8e74-08fa7e3aba92 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:46.119353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db82a8-7ede-439c-8e74-08fa7e3aba92 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:46.119628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db82a8-7ede-439c-8e74-08fa7e3aba92 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:46.128192 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db82a8-7ede-439c-8e74-08fa7e3aba92 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:46.130247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db82a8-7ede-439c-8e74-08fa7e3aba92 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-c1db82a8-7ede-439c-8e74-08fa7e3aba92 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:46.142519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:46.143688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:46.145273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:46.145273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:46.145273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:46.145273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:46.145273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:46.145273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:46.145273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:46.145273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:46.145273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:01:46.145273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:46.146493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 512/1027] 10.222.0.100 () {56 vars in 1098 bytes} [Mon Aug 1 15:01:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:46.154062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a529f-da3d-4da6-a99d-345c1afbbc9f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:46.271082 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a529f-da3d-4da6-a99d-345c1afbbc9f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:46.307039 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53c3fb-d100-4f88-8a7e-cb696e0a9305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:46.307722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c3fb-d100-4f88-8a7e-cb696e0a9305 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:46.307940 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c3fb-d100-4f88-8a7e-cb696e0a9305 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:46.308195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c3fb-d100-4f88-8a7e-cb696e0a9305 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:46.309239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53c3fb-d100-4f88-8a7e-cb696e0a9305 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:46.309793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53c3fb-d100-4f88-8a7e-cb696e0a9305 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:46.311443 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53c3fb-d100-4f88-8a7e-cb696e0a9305 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:46.327379 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:46.337562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 516/1028] 10.222.0.100 () {56 vars in 1138 bytes} [Mon Aug 1 15:01:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:46.338906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db82a8-7ede-439c-8e74-08fa7e3aba92 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:46.444557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:46.448578 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db82a8-7ede-439c-8e74-08fa7e3aba92 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:46.484371 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d6229-569e-47ea-9c8c-a1102adcb4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:46.485002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d6229-569e-47ea-9c8c-a1102adcb4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:46.485282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d6229-569e-47ea-9c8c-a1102adcb4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:46.485556 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d6229-569e-47ea-9c8c-a1102adcb4a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:46.579202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d6229-569e-47ea-9c8c-a1102adcb4a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:46.580707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d6229-569e-47ea-9c8c-a1102adcb4a1 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:46.691192 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:46.702781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 513/1029] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:46 2022] GET /identity/v3/auth/tokens => generated 1963 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:01:46.704202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53c3fb-d100-4f88-8a7e-cb696e0a9305 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:46.737187 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ad41a-dfcc-4c50-9eb6-90708699474f None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:46.737536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ad41a-dfcc-4c50-9eb6-90708699474f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:46.737754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ad41a-dfcc-4c50-9eb6-90708699474f None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:46.739334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 514/1030] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:46.744284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1eec7-6402-4b5d-a8b0-aa0ecc1c3f94 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:46.744574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1eec7-6402-4b5d-a8b0-aa0ecc1c3f94 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:46.744855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db1eec7-6402-4b5d-a8b0-aa0ecc1c3f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:46.763930 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0db1eec7-6402-4b5d-a8b0-aa0ecc1c3f94 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:46.765151 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0db1eec7-6402-4b5d-a8b0-aa0ecc1c3f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:46.815389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53c3fb-d100-4f88-8a7e-cb696e0a9305 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:46.838791 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 515/1031] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:46 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:46.854406 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc76cc-3384-425d-bc62-df4b68ce2f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:46.855089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc76cc-3384-425d-bc62-df4b68ce2f9e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:46.855385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc76cc-3384-425d-bc62-df4b68ce2f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:46.855720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc76cc-3384-425d-bc62-df4b68ce2f9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:46.863340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1eec7-6402-4b5d-a8b0-aa0ecc1c3f94 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:46.866992 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc76cc-3384-425d-bc62-df4b68ce2f9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:46.868461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc76cc-3384-425d-bc62-df4b68ce2f9e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-04cc76cc-3384-425d-bc62-df4b68ce2f9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:46.884240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:46.885226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:46.891056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:46.891056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:46.891056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:46.891056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:46.891056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 517/1032] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:46.891056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc76cc-3384-425d-bc62-df4b68ce2f9e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:46.950053 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:46.981863 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db1eec7-6402-4b5d-a8b0-aa0ecc1c3f94 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:47.004125 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc76cc-3384-425d-bc62-df4b68ce2f9e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:47.013700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:47.024488 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1af714-01aa-4245-8ec6-c3f0a48596fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:47.025159 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1af714-01aa-4245-8ec6-c3f0a48596fb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:47.025466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1af714-01aa-4245-8ec6-c3f0a48596fb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:47.025711 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1af714-01aa-4245-8ec6-c3f0a48596fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:47.027428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1af714-01aa-4245-8ec6-c3f0a48596fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:47.028893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1af714-01aa-4245-8ec6-c3f0a48596fb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:47.030711 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 516/1033] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:46 2022] GET /identity/v3/domains/default => generated 200 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:47.037274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1af714-01aa-4245-8ec6-c3f0a48596fb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:47.042881 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4767f-2793-4b3e-ace6-b02304618a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:47.043558 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4767f-2793-4b3e-ace6-b02304618a06 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:47.043845 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4767f-2793-4b3e-ace6-b02304618a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:47.044131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4767f-2793-4b3e-ace6-b02304618a06 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:47.044913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4767f-2793-4b3e-ace6-b02304618a06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:47.045224 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4767f-2793-4b3e-ace6-b02304618a06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:47.046680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4767f-2793-4b3e-ace6-b02304618a06 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:47.061388 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 518/1034] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:47.067657 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4767f-2793-4b3e-ace6-b02304618a06 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:47.156664 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1af714-01aa-4245-8ec6-c3f0a48596fb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:47.195313 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4767f-2793-4b3e-ace6-b02304618a06 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:47.195593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2dcde7-4e26-4899-9cf8-73c505feee87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:47.196273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2dcde7-4e26-4899-9cf8-73c505feee87 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:47.196558 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2dcde7-4e26-4899-9cf8-73c505feee87 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:47.196842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2dcde7-4e26-4899-9cf8-73c505feee87 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:47.197537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2dcde7-4e26-4899-9cf8-73c505feee87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:47.199014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2dcde7-4e26-4899-9cf8-73c505feee87 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:47.215047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:47.225546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 517/1035] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:01:47 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:01:47.260163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec6448-f877-4b03-b491-78f9ebe87f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:47.260754 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec6448-f877-4b03-b491-78f9ebe87f60 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:47.261037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec6448-f877-4b03-b491-78f9ebe87f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:47.261316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec6448-f877-4b03-b491-78f9ebe87f60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:47.269962 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec6448-f877-4b03-b491-78f9ebe87f60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:47.271491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec6448-f877-4b03-b491-78f9ebe87f60 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-90ec6448-f877-4b03-b491-78f9ebe87f60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:47.284529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:47.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:47.286543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:47.286543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:47.286543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:47.286543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:47.286543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:47.286543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:47.286543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:47.286543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:47.286543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 519/1036] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:47.291778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2dcde7-4e26-4899-9cf8-73c505feee87 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:47.366183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:47.424405 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2dcde7-4e26-4899-9cf8-73c505feee87 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:47.467064 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f8580-7542-44f4-b180-842399f36d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:47.467884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f8580-7542-44f4-b180-842399f36d39 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:47.468265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f8580-7542-44f4-b180-842399f36d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:47.469281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f8580-7542-44f4-b180-842399f36d39 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:47.471615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f8580-7542-44f4-b180-842399f36d39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:47.471615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f8580-7542-44f4-b180-842399f36d39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:47.471615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f8580-7542-44f4-b180-842399f36d39 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:47.486743 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 518/1037] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:47.493607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec6448-f877-4b03-b491-78f9ebe87f60 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:47.496931 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac437a56-5641-410b-8592-cc5e748a18f3 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:47.497227 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac437a56-5641-410b-8592-cc5e748a18f3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:47.497509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac437a56-5641-410b-8592-cc5e748a18f3 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:47.499065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 519/1038] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:01:47.504900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96800710-d860-4b29-aa7a-c2d59d86b2c2 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:47.505206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96800710-d860-4b29-aa7a-c2d59d86b2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:47.505467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96800710-d860-4b29-aa7a-c2d59d86b2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:47.525708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-96800710-d860-4b29-aa7a-c2d59d86b2c2 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:47.526887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-96800710-d860-4b29-aa7a-c2d59d86b2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:47.614716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 520/1039] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:47 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:47.624063 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec6448-f877-4b03-b491-78f9ebe87f60 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:47.637039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96800710-d860-4b29-aa7a-c2d59d86b2c2 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:47.664666 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd2d37-1743-4266-9a60-1f73380100f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:47.665394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd2d37-1743-4266-9a60-1f73380100f6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:47.665751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd2d37-1743-4266-9a60-1f73380100f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:47.666080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd2d37-1743-4266-9a60-1f73380100f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:47.666989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd2d37-1743-4266-9a60-1f73380100f6 None admin] RBAC: Adding query filter params (role.id=46f67f665d95448798ec8aef267e78fb, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=ec50ac7c54324884a1b2a6c63d617907) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:47.667375 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd2d37-1743-4266-9a60-1f73380100f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:47.668901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd2d37-1743-4266-9a60-1f73380100f6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:47.674836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 520/1040] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:47 2022] GET /identity/v3/role_assignments?user.id=ec50ac7c54324884a1b2a6c63d617907&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=46f67f665d95448798ec8aef267e78fb => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:47.706994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:47.778375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96800710-d860-4b29-aa7a-c2d59d86b2c2 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:47.819486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef741a-02b1-4980-9364-a1e3f13be189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:47.820259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef741a-02b1-4980-9364-a1e3f13be189 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:47.820620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef741a-02b1-4980-9364-a1e3f13be189 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:47.821030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef741a-02b1-4980-9364-a1e3f13be189 None admin] PATH_INFO: `/v3/services/image` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:47.821758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef741a-02b1-4980-9364-a1e3f13be189 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:47.823333 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef741a-02b1-4980-9364-a1e3f13be189 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-f2ef741a-02b1-4980-9364-a1e3f13be189 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:47.833547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:47.834895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:47.835951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 521/1041] 10.222.0.100 () {56 vars in 1110 bytes} [Mon Aug 1 15:01:47 2022] GET /identity/v3/services/image => generated 86 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:47.842640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd2d37-1743-4266-9a60-1f73380100f6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:47.887935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:47.965765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd2d37-1743-4266-9a60-1f73380100f6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:48.005320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d3bbf-da67-4f52-8eae-dc844f19eaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:48.006095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d3bbf-da67-4f52-8eae-dc844f19eaed None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:48.006449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d3bbf-da67-4f52-8eae-dc844f19eaed None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:48.006855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d3bbf-da67-4f52-8eae-dc844f19eaed None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:48.007619 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d3bbf-da67-4f52-8eae-dc844f19eaed None admin] RBAC: Adding query filter params (name=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:48.008060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d3bbf-da67-4f52-8eae-dc844f19eaed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:48.009517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d3bbf-da67-4f52-8eae-dc844f19eaed None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:48.019936 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 521/1042] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:47 2022] GET /identity/v3/services?name=image => generated 123 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:48.021563 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef741a-02b1-4980-9364-a1e3f13be189 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:48.146922 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef741a-02b1-4980-9364-a1e3f13be189 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:48.157527 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:48.189716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e937c7b-67ed-4e7b-bdf7-869f3e827808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:48.190517 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e937c7b-67ed-4e7b-bdf7-869f3e827808 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:48.190982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e937c7b-67ed-4e7b-bdf7-869f3e827808 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:48.191418 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e937c7b-67ed-4e7b-bdf7-869f3e827808 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:48.192210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e937c7b-67ed-4e7b-bdf7-869f3e827808 None admin] RBAC: Adding query filter params (type=image) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:48.192611 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e937c7b-67ed-4e7b-bdf7-869f3e827808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:48.194085 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e937c7b-67ed-4e7b-bdf7-869f3e827808 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:48.204566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 522/1043] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:48 2022] GET /identity/v3/services?type=image => generated 352 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:48.205711 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d3bbf-da67-4f52-8eae-dc844f19eaed None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:48.320952 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:48.325343 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d3bbf-da67-4f52-8eae-dc844f19eaed None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:48.361930 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ae3e4-068d-4a21-89f9-2f74d6f474d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:48.362544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ae3e4-068d-4a21-89f9-2f74d6f474d7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:48.362840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ae3e4-068d-4a21-89f9-2f74d6f474d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:48.363654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ae3e4-068d-4a21-89f9-2f74d6f474d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:48.364412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ae3e4-068d-4a21-89f9-2f74d6f474d7 None admin] RBAC: Adding query filter params (interface=public, service_id=e248a9d4c86b49dca6b5ea0897d6b6e3, region_id=RegionOne) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:48.364720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ae3e4-068d-4a21-89f9-2f74d6f474d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:48.366154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ae3e4-068d-4a21-89f9-2f74d6f474d7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:48.372282 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 522/1044] 10.222.0.100 () {56 vars in 1256 bytes} [Mon Aug 1 15:01:48 2022] GET /identity/v3/endpoints?service_id=e248a9d4c86b49dca6b5ea0897d6b6e3&interface=public®ion_id=RegionOne => generated 195 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:48.377854 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e937c7b-67ed-4e7b-bdf7-869f3e827808 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:48.490005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e937c7b-67ed-4e7b-bdf7-869f3e827808 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:48.533525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c31966-9be7-480e-8ed7-7d17c0068f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:48.534157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c31966-9be7-480e-8ed7-7d17c0068f36 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:48.534427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c31966-9be7-480e-8ed7-7d17c0068f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:48.539757 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c31966-9be7-480e-8ed7-7d17c0068f36 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:48.539757 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c31966-9be7-480e-8ed7-7d17c0068f36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:48.539757 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c31966-9be7-480e-8ed7-7d17c0068f36 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:48.553800 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 523/1045] 10.222.0.100 () {56 vars in 1092 bytes} [Mon Aug 1 15:01:48 2022] GET /identity/v3/services => generated 1966 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:01:49.435458 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8298879-6f89-44b3-baf3-e8c9de19b412 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:49.435996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8298879-6f89-44b3-baf3-e8c9de19b412 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:49.435996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8298879-6f89-44b3-baf3-e8c9de19b412 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:49.437423 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 523/1046] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:49.443570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea6594-7b59-47e9-abf5-495f3e669899 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:49.444066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea6594-7b59-47e9-abf5-495f3e669899 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:49.444150 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea6594-7b59-47e9-abf5-495f3e669899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:49.464222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3dea6594-7b59-47e9-abf5-495f3e669899 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:49.465581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3dea6594-7b59-47e9-abf5-495f3e669899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:49.501670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:49.558235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 524/1047] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:49 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:49.580252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8298879-6f89-44b3-baf3-e8c9de19b412 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:49.718085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8298879-6f89-44b3-baf3-e8c9de19b412 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:49.727433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc796a-9c55-42a3-9c4d-a8b4478c2edc None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:49.727806 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc796a-9c55-42a3-9c4d-a8b4478c2edc None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:49.728000 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc796a-9c55-42a3-9c4d-a8b4478c2edc None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:49.729509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 525/1048] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:01:49.734868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdad36c-8f03-4846-9552-957024aa3f61 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:49.735218 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdad36c-8f03-4846-9552-957024aa3f61 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:49.735442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdad36c-8f03-4846-9552-957024aa3f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:49.754449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3cdad36c-8f03-4846-9552-957024aa3f61 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:49.755816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3cdad36c-8f03-4846-9552-957024aa3f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:49.758022 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd31e7-53fa-481c-a270-46b4cd632609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:49.758702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd31e7-53fa-481c-a270-46b4cd632609 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:49.759000 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd31e7-53fa-481c-a270-46b4cd632609 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:49.760306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd31e7-53fa-481c-a270-46b4cd632609 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:49.772514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd31e7-53fa-481c-a270-46b4cd632609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-38bd31e7-53fa-481c-a270-46b4cd632609 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:49.777505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:49.778505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:49.779895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:49.779895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:49.779895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:49.779895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:49.779895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:49.779895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:49.779895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:49.779895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:49.779895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:49.779895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:49.779895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:49.779895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:49.779895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 524/1049] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:01:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:49.786368 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd31e7-53fa-481c-a270-46b4cd632609 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:49.815316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:49.841957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 526/1050] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:49 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:49.864564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdad36c-8f03-4846-9552-957024aa3f61 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:49.904039 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd31e7-53fa-481c-a270-46b4cd632609 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:49.947753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a80135-6233-4db8-bf9d-296543100e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:49.948385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a80135-6233-4db8-bf9d-296543100e98 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:49.948810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a80135-6233-4db8-bf9d-296543100e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:49.949091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a80135-6233-4db8-bf9d-296543100e98 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:49.949828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a80135-6233-4db8-bf9d-296543100e98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:49.950142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a80135-6233-4db8-bf9d-296543100e98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:49.951576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a80135-6233-4db8-bf9d-296543100e98 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:49.967428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 525/1051] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:01:49 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:49.968551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a80135-6233-4db8-bf9d-296543100e98 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:49.972508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:49.980576 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdad36c-8f03-4846-9552-957024aa3f61 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:50.023627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a264740-4313-4939-b8f0-b0e086e375a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:50.024292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a264740-4313-4939-b8f0-b0e086e375a5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:50.024576 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a264740-4313-4939-b8f0-b0e086e375a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:50.024848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a264740-4313-4939-b8f0-b0e086e375a5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:50.036286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a264740-4313-4939-b8f0-b0e086e375a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-2a264740-4313-4939-b8f0-b0e086e375a5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:50.041235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:50.042147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:50.047619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:50.047619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:50.047619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:50.047619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:50.047619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:50.047619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:50.047619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:50.047619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:50.047619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:50.047619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:50.047619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:01:50.047619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:50.047619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 527/1052] 10.222.0.100 () {56 vars in 1125 bytes} [Mon Aug 1 15:01:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:50.049041 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a264740-4313-4939-b8f0-b0e086e375a5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:50.081400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:50.099826 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a80135-6233-4db8-bf9d-296543100e98 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:50.137766 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e74b84-6045-4f96-9f53-6e0756f2e1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:50.138422 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e74b84-6045-4f96-9f53-6e0756f2e1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:50.138723 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e74b84-6045-4f96-9f53-6e0756f2e1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:50.139003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e74b84-6045-4f96-9f53-6e0756f2e1f1 None admin] PATH_INFO: `/v3/users/fbc2d9a22c934d7598d887aa49393419` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:50.154724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e74b84-6045-4f96-9f53-6e0756f2e1f1 None admin] RBAC: Authorizing `identity:get_user(user_id=fbc2d9a22c934d7598d887aa49393419)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:50.156204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e74b84-6045-4f96-9f53-6e0756f2e1f1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:50.162691 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 526/1053] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:01:49 2022] GET /identity/v3/users/fbc2d9a22c934d7598d887aa49393419 => generated 286 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:50.163798 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e74b84-6045-4f96-9f53-6e0756f2e1f1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:50.177880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a264740-4313-4939-b8f0-b0e086e375a5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:50.217416 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf4e8d-5e36-4975-be42-ad71a414f918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:50.218053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf4e8d-5e36-4975-be42-ad71a414f918 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:50.218334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf4e8d-5e36-4975-be42-ad71a414f918 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:50.218755 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf4e8d-5e36-4975-be42-ad71a414f918 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:50.219385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf4e8d-5e36-4975-be42-ad71a414f918 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:50.219689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf4e8d-5e36-4975-be42-ad71a414f918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:50.221145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf4e8d-5e36-4975-be42-ad71a414f918 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:50.235533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 528/1054] 10.222.0.100 () {56 vars in 1120 bytes} [Mon Aug 1 15:01:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:50.236348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf4e8d-5e36-4975-be42-ad71a414f918 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:50.258734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:50.292412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e74b84-6045-4f96-9f53-6e0756f2e1f1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:50.332568 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af29f1e-57bc-427f-a022-8fe7f039d7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:50.333202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af29f1e-57bc-427f-a022-8fe7f039d7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:50.333483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af29f1e-57bc-427f-a022-8fe7f039d7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:50.333768 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af29f1e-57bc-427f-a022-8fe7f039d7b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:50.349354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:50.349969 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf4e8d-5e36-4975-be42-ad71a414f918 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:50.391482 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cea040-a011-4421-9334-b9064a0f4b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:50.392134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cea040-a011-4421-9334-b9064a0f4b19 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:50.392418 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cea040-a011-4421-9334-b9064a0f4b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:50.392696 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cea040-a011-4421-9334-b9064a0f4b19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:50.404036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cea040-a011-4421-9334-b9064a0f4b19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:50.405503 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cea040-a011-4421-9334-b9064a0f4b19 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:50.419140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af29f1e-57bc-427f-a022-8fe7f039d7b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-41cea040-a011-4421-9334-b9064a0f4b19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:50.420167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:50.421070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:50.422003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:50.422003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:50.422003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:50.422003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:50.422003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 529/1055] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:50.422774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af29f1e-57bc-427f-a022-8fe7f039d7b4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:50.427857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cea040-a011-4421-9334-b9064a0f4b19 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:50.513719 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:50.552431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 527/1056] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:50 2022] GET /identity/v3/auth/tokens => generated 1963 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:01:50.555986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cea040-a011-4421-9334-b9064a0f4b19 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:50.558160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af29f1e-57bc-427f-a022-8fe7f039d7b4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:50.600155 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d17e5-b2ee-42ee-802e-548d82a57e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:50.600818 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d17e5-b2ee-42ee-802e-548d82a57e4f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:50.601103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d17e5-b2ee-42ee-802e-548d82a57e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:50.601378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d17e5-b2ee-42ee-802e-548d82a57e4f None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:50.602079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d17e5-b2ee-42ee-802e-548d82a57e4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:50.602384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d17e5-b2ee-42ee-802e-548d82a57e4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:50.603812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d17e5-b2ee-42ee-802e-548d82a57e4f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:50.617320 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 530/1057] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:50.618017 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:50.623174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d17e5-b2ee-42ee-802e-548d82a57e4f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:50.691472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af29f1e-57bc-427f-a022-8fe7f039d7b4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:50.727525 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34ea37-35be-44cb-baf1-5aad0c2fcf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:50.728181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34ea37-35be-44cb-baf1-5aad0c2fcf5e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:50.728466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34ea37-35be-44cb-baf1-5aad0c2fcf5e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:50.728741 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34ea37-35be-44cb-baf1-5aad0c2fcf5e None admin] PATH_INFO: `/v3/projects/1454aaf365d148c3a1cc9caec965e79e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:50.729927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34ea37-35be-44cb-baf1-5aad0c2fcf5e None admin] RBAC: Authorizing `identity:get_project(project_id=1454aaf365d148c3a1cc9caec965e79e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:50.731393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d17e5-b2ee-42ee-802e-548d82a57e4f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:50.731536 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34ea37-35be-44cb-baf1-5aad0c2fcf5e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:50.733110 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 528/1058] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:01:50 2022] GET /identity/v3/projects/1454aaf365d148c3a1cc9caec965e79e => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:50.734289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981bf1a-b444-4e8d-ad76-79104d809357 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:50.734578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981bf1a-b444-4e8d-ad76-79104d809357 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:50.734884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981bf1a-b444-4e8d-ad76-79104d809357 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:50.736109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 529/1059] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:50 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:50.739713 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981bf1a-b444-4e8d-ad76-79104d809357 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:50.771727 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:50.777996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc62c0-3b62-46db-8417-b79dd96174de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:50.778652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc62c0-3b62-46db-8417-b79dd96174de None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:50.779077 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc62c0-3b62-46db-8417-b79dd96174de None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:50.779356 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc62c0-3b62-46db-8417-b79dd96174de None admin] PATH_INFO: `/v3/users/nova` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:50.787507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc62c0-3b62-46db-8417-b79dd96174de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:50.788963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc62c0-3b62-46db-8417-b79dd96174de None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-1acc62c0-3b62-46db-8417-b79dd96174de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:50.802261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:50.803698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:50.805152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:50.805152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:50.805152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:50.805152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:50.805152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:50.805152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:50.805152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:50.805152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:50.805152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:01:50.805152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:50.806828 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 531/1060] 10.222.0.100 () {56 vars in 1098 bytes} [Mon Aug 1 15:01:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:50.808444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3427db-e72f-4b02-a8d6-5fb612b62ff6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:50.808897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3427db-e72f-4b02-a8d6-5fb612b62ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:50.809187 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3427db-e72f-4b02-a8d6-5fb612b62ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:50.829389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-bc3427db-e72f-4b02-a8d6-5fb612b62ff6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:50.830622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bc3427db-e72f-4b02-a8d6-5fb612b62ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:50.859516 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981bf1a-b444-4e8d-ad76-79104d809357 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:50.873706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:50.901542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc02c3d-51d5-4e76-9f91-5953cda39de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:50.902346 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc02c3d-51d5-4e76-9f91-5953cda39de4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:50.902515 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc02c3d-51d5-4e76-9f91-5953cda39de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:50.902774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc02c3d-51d5-4e76-9f91-5953cda39de4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:50.903228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 532/1061] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:50 2022] POST /identity/v3/auth/tokens => generated 1963 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:50.903587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc02c3d-51d5-4e76-9f91-5953cda39de4 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=1454aaf365d148c3a1cc9caec965e79e, user.id=fbc2d9a22c934d7598d887aa49393419) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:50.903904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc02c3d-51d5-4e76-9f91-5953cda39de4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:50.904088 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3427db-e72f-4b02-a8d6-5fb612b62ff6 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:50.905478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc02c3d-51d5-4e76-9f91-5953cda39de4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:50.911857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 530/1062] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:50 2022] GET /identity/v3/role_assignments?user.id=fbc2d9a22c934d7598d887aa49393419&scope.project.id=1454aaf365d148c3a1cc9caec965e79e&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:50.931580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc02c3d-51d5-4e76-9f91-5953cda39de4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:51.046701 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3427db-e72f-4b02-a8d6-5fb612b62ff6 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:51.057355 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc02c3d-51d5-4e76-9f91-5953cda39de4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:51.075598 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:51.098741 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39a55b-3876-45d9-8c8f-2e561988431e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:51.098986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5a828-da30-4d01-9baf-e204f528b8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:51.099530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39a55b-3876-45d9-8c8f-2e561988431e None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:51.099658 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5a828-da30-4d01-9baf-e204f528b8fd None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:51.099815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39a55b-3876-45d9-8c8f-2e561988431e None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:51.099923 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5a828-da30-4d01-9baf-e204f528b8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:51.100098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39a55b-3876-45d9-8c8f-2e561988431e None admin] PATH_INFO: `/v3/services/image` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:51.100210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5a828-da30-4d01-9baf-e204f528b8fd None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:51.100736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39a55b-3876-45d9-8c8f-2e561988431e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:51.100906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5a828-da30-4d01-9baf-e204f528b8fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:51.101222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5a828-da30-4d01-9baf-e204f528b8fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:51.102133 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39a55b-3876-45d9-8c8f-2e561988431e None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:51.102783 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5a828-da30-4d01-9baf-e204f528b8fd None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3e39a55b-3876-45d9-8c8f-2e561988431e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:51.117030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:51.118143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:51.119478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:51.119478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:51.119478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:51.119478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:51.119478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:51.119478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:51.119478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:51.119478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:51.119478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:51.119478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:51.119478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:01:51.119478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:01:51.119478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 15:01:51.119478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:51.122979 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 531/1063] 10.222.0.100 () {56 vars in 1110 bytes} [Mon Aug 1 15:01:50 2022] GET /identity/v3/services/image => generated 86 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:51.124358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39a55b-3876-45d9-8c8f-2e561988431e None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:51.129317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 533/1064] 10.222.0.100 () {56 vars in 1138 bytes} [Mon Aug 1 15:01:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:51.130231 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5a828-da30-4d01-9baf-e204f528b8fd None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:51.182234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:51.255586 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5a828-da30-4d01-9baf-e204f528b8fd None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:51.261153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39a55b-3876-45d9-8c8f-2e561988431e None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:51.299284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b0794-1be1-4a94-b0a8-d81416b4c46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:51.299857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b0794-1be1-4a94-b0a8-d81416b4c46a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:51.300143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b0794-1be1-4a94-b0a8-d81416b4c46a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:51.300453 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b0794-1be1-4a94-b0a8-d81416b4c46a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:51.300886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea1c78-fed0-481e-88b6-26008ff17b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:51.301743 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea1c78-fed0-481e-88b6-26008ff17b76 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:51.302047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea1c78-fed0-481e-88b6-26008ff17b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:51.302331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea1c78-fed0-481e-88b6-26008ff17b76 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:51.303065 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea1c78-fed0-481e-88b6-26008ff17b76 None admin] RBAC: Adding query filter params (name=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:51.303434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea1c78-fed0-481e-88b6-26008ff17b76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:51.304835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea1c78-fed0-481e-88b6-26008ff17b76 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:51.315267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 532/1065] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:51 2022] GET /identity/v3/services?name=image => generated 123 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:51.315462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b0794-1be1-4a94-b0a8-d81416b4c46a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:51.317334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea1c78-fed0-481e-88b6-26008ff17b76 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:51.317450 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b0794-1be1-4a94-b0a8-d81416b4c46a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-298b0794-1be1-4a94-b0a8-d81416b4c46a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:51.334805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:51.335981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:51.336913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:51.336913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:51.336913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:51.336913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:51.336913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 534/1066] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:51.342519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b0794-1be1-4a94-b0a8-d81416b4c46a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:51.391166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:51.446368 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea1c78-fed0-481e-88b6-26008ff17b76 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:51.448032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:51.478113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b0794-1be1-4a94-b0a8-d81416b4c46a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:51.486447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05253c6-79b3-4cf1-86dd-296d6abd2fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:51.487374 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05253c6-79b3-4cf1-86dd-296d6abd2fbc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:51.487794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05253c6-79b3-4cf1-86dd-296d6abd2fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:51.488174 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05253c6-79b3-4cf1-86dd-296d6abd2fbc None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:51.489187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05253c6-79b3-4cf1-86dd-296d6abd2fbc None admin] RBAC: Adding query filter params (type=image) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:51.489494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05253c6-79b3-4cf1-86dd-296d6abd2fbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:51.491127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05253c6-79b3-4cf1-86dd-296d6abd2fbc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:51.497237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 533/1067] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:51 2022] GET /identity/v3/services?type=image => generated 352 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:51.502635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05253c6-79b3-4cf1-86dd-296d6abd2fbc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:51.518034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373d6bc-17ec-4aec-9cb7-3cbaf72e1a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:51.518812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373d6bc-17ec-4aec-9cb7-3cbaf72e1a35 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:51.519594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373d6bc-17ec-4aec-9cb7-3cbaf72e1a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:51.519868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373d6bc-17ec-4aec-9cb7-3cbaf72e1a35 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:51.520568 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373d6bc-17ec-4aec-9cb7-3cbaf72e1a35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:51.520876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373d6bc-17ec-4aec-9cb7-3cbaf72e1a35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:51.522367 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373d6bc-17ec-4aec-9cb7-3cbaf72e1a35 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:51.535834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 535/1068] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:51.545266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373d6bc-17ec-4aec-9cb7-3cbaf72e1a35 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:51.627009 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05253c6-79b3-4cf1-86dd-296d6abd2fbc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:51.663346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:51.671324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401412c-71fa-420a-9a2a-d0f421a873ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:51.674329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401412c-71fa-420a-9a2a-d0f421a873ea None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:51.674329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401412c-71fa-420a-9a2a-d0f421a873ea None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:51.674329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401412c-71fa-420a-9a2a-d0f421a873ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:51.674329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401412c-71fa-420a-9a2a-d0f421a873ea None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:51.674755 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401412c-71fa-420a-9a2a-d0f421a873ea None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:51.678638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373d6bc-17ec-4aec-9cb7-3cbaf72e1a35 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:51.708364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 534/1069] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:01:51 2022] POST /identity/v3/endpoints => generated 329 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:01:51.731964 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b9dd9-d166-46ab-a6d9-2ab37991dea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:51.732743 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b9dd9-d166-46ab-a6d9-2ab37991dea8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:51.733033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b9dd9-d166-46ab-a6d9-2ab37991dea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:51.733307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b9dd9-d166-46ab-a6d9-2ab37991dea8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:51.742196 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b9dd9-d166-46ab-a6d9-2ab37991dea8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:51.743705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b9dd9-d166-46ab-a6d9-2ab37991dea8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-d56b9dd9-d166-46ab-a6d9-2ab37991dea8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:51.757883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:51.759144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:51.760122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:51.760122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:51.760122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:51.760122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:51.760122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:51.760122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:51.760122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:51.760122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:51.763070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 536/1070] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:51.765394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401412c-71fa-420a-9a2a-d0f421a873ea None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:51.796916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:51.902287 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401412c-71fa-420a-9a2a-d0f421a873ea None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:51.937928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86775de-331c-49b4-a7c6-d1884064a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:51.938748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86775de-331c-49b4-a7c6-d1884064a210 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:51.939136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86775de-331c-49b4-a7c6-d1884064a210 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:51.939519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86775de-331c-49b4-a7c6-d1884064a210 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:51.940383 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86775de-331c-49b4-a7c6-d1884064a210 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:51.940797 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86775de-331c-49b4-a7c6-d1884064a210 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:51.942988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86775de-331c-49b4-a7c6-d1884064a210 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:51.956667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 535/1071] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:51.963203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b9dd9-d166-46ab-a6d9-2ab37991dea8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:52.084106 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b9dd9-d166-46ab-a6d9-2ab37991dea8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:52.129674 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828882c-0ea2-49d8-b500-1f722a6cd63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:52.130625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828882c-0ea2-49d8-b500-1f722a6cd63c None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:52.131280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828882c-0ea2-49d8-b500-1f722a6cd63c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:52.131907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828882c-0ea2-49d8-b500-1f722a6cd63c None admin] PATH_INFO: `/v3/projects/d6e147b6607b481591ceb250f7208c16/users/ec50ac7c54324884a1b2a6c63d617907/roles/46f67f665d95448798ec8aef267e78fb` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:52.134466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828882c-0ea2-49d8-b500-1f722a6cd63c None admin] RBAC: Authorizing `identity:create_grant(project_id=d6e147b6607b481591ceb250f7208c16, user_id=ec50ac7c54324884a1b2a6c63d617907, role_id=46f67f665d95448798ec8aef267e78fb)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:52.136723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828882c-0ea2-49d8-b500-1f722a6cd63c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:52.156288 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 537/1072] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:01:51 2022] PUT /identity/v3/projects/d6e147b6607b481591ceb250f7208c16/users/ec50ac7c54324884a1b2a6c63d617907/roles/46f67f665d95448798ec8aef267e78fb => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:01:52.985492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82732ef4-db53-4e48-be36-4ae418eb681d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:52.986093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82732ef4-db53-4e48-be36-4ae418eb681d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:52.986093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82732ef4-db53-4e48-be36-4ae418eb681d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:52.989468 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 536/1073] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:52.995478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070d5dd-07bb-46a8-b8a0-3258ca533908 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:52.996082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070d5dd-07bb-46a8-b8a0-3258ca533908 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:52.996258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f070d5dd-07bb-46a8-b8a0-3258ca533908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:53.008594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:53.016942 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f070d5dd-07bb-46a8-b8a0-3258ca533908 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:53.018740 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f070d5dd-07bb-46a8-b8a0-3258ca533908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:53.094279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 538/1074] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:52 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:53.117683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82732ef4-db53-4e48-be36-4ae418eb681d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:53.212937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:53.236558 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82732ef4-db53-4e48-be36-4ae418eb681d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:53.276958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b918897-8fed-4c93-b05a-76217e92d317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:53.277628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b918897-8fed-4c93-b05a-76217e92d317 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:53.277909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b918897-8fed-4c93-b05a-76217e92d317 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:53.278180 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b918897-8fed-4c93-b05a-76217e92d317 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:53.289862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b918897-8fed-4c93-b05a-76217e92d317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8b918897-8fed-4c93-b05a-76217e92d317 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:53.295290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:53.296288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:53.297161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:53.297161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:53.297161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:53.297161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:53.297161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:53.297161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:53.297161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:53.297161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:53.297161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:53.297161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:53.297161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:53.297161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:53.298737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 537/1075] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:01:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:53.302623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070d5dd-07bb-46a8-b8a0-3258ca533908 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:53.411134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f070d5dd-07bb-46a8-b8a0-3258ca533908 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:53.450548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15176227-e574-4c9f-bad2-9903b7d1e65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:53.451200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15176227-e574-4c9f-bad2-9903b7d1e65f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:53.451479 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15176227-e574-4c9f-bad2-9903b7d1e65f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:53.451741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15176227-e574-4c9f-bad2-9903b7d1e65f None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:53.452469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15176227-e574-4c9f-bad2-9903b7d1e65f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:53.452773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15176227-e574-4c9f-bad2-9903b7d1e65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:53.454156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15176227-e574-4c9f-bad2-9903b7d1e65f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:53.457283 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:53.470833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 539/1076] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:01:53 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:53.476825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b918897-8fed-4c93-b05a-76217e92d317 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:53.606195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b918897-8fed-4c93-b05a-76217e92d317 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:53.647070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61123e81-f9e1-4bac-8244-386390b4af16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:53.647729 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61123e81-f9e1-4bac-8244-386390b4af16 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:53.648014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61123e81-f9e1-4bac-8244-386390b4af16 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:53.648294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61123e81-f9e1-4bac-8244-386390b4af16 None admin] PATH_INFO: `/v3/users/fbc2d9a22c934d7598d887aa49393419` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:53.649553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61123e81-f9e1-4bac-8244-386390b4af16 None admin] RBAC: Authorizing `identity:get_user(user_id=fbc2d9a22c934d7598d887aa49393419)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:53.651057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61123e81-f9e1-4bac-8244-386390b4af16 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:53.654873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 538/1077] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:01:53 2022] GET /identity/v3/users/fbc2d9a22c934d7598d887aa49393419 => generated 286 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:53.658515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15176227-e574-4c9f-bad2-9903b7d1e65f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:53.776279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15176227-e574-4c9f-bad2-9903b7d1e65f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:53.789041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4404019-562d-4e78-8ae5-d348db9a1b2c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:53.789382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4404019-562d-4e78-8ae5-d348db9a1b2c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:53.789608 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4404019-562d-4e78-8ae5-d348db9a1b2c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:53.791012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 539/1078] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:01:53.796294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f4651-1dfa-4c35-a5aa-0a020d60dc1d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:53.796590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f4651-1dfa-4c35-a5aa-0a020d60dc1d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:53.796875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f4651-1dfa-4c35-a5aa-0a020d60dc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:53.814745 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d07b7-83a3-4463-8c8a-78160bb1258d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:53.815322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d07b7-83a3-4463-8c8a-78160bb1258d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:53.815606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d07b7-83a3-4463-8c8a-78160bb1258d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:53.816558 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d07b7-83a3-4463-8c8a-78160bb1258d None admin] PATH_INFO: `/v3/projects/1454aaf365d148c3a1cc9caec965e79e` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:53.817772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d07b7-83a3-4463-8c8a-78160bb1258d None admin] RBAC: Authorizing `identity:get_project(project_id=1454aaf365d148c3a1cc9caec965e79e)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:53.819264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d07b7-83a3-4463-8c8a-78160bb1258d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:53.820977 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 540/1079] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:01:53 2022] GET /identity/v3/projects/1454aaf365d148c3a1cc9caec965e79e => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:53.828171 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-689f4651-1dfa-4c35-a5aa-0a020d60dc1d None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:53.829405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-689f4651-1dfa-4c35-a5aa-0a020d60dc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:53.831682 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d07b7-83a3-4463-8c8a-78160bb1258d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:53.838742 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:53.911544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:53.918983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 540/1080] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:53 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:01:53.934394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f4651-1dfa-4c35-a5aa-0a020d60dc1d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:53.953961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d07b7-83a3-4463-8c8a-78160bb1258d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:53.993175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66b2a1-e77b-40c9-bb9b-e56e476cd550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:53.993975 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66b2a1-e77b-40c9-bb9b-e56e476cd550 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:53.994265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66b2a1-e77b-40c9-bb9b-e56e476cd550 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:53.994654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66b2a1-e77b-40c9-bb9b-e56e476cd550 None admin] PATH_INFO: `/v3/projects/1454aaf365d148c3a1cc9caec965e79e/users/fbc2d9a22c934d7598d887aa49393419/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:53.996683 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66b2a1-e77b-40c9-bb9b-e56e476cd550 None admin] RBAC: Authorizing `identity:create_grant(project_id=1454aaf365d148c3a1cc9caec965e79e, user_id=fbc2d9a22c934d7598d887aa49393419, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:53.998446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66b2a1-e77b-40c9-bb9b-e56e476cd550 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:54.017877 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 541/1081] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:01:53 2022] PUT /identity/v3/projects/1454aaf365d148c3a1cc9caec965e79e/users/fbc2d9a22c934d7598d887aa49393419/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:01:54.053146 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f4651-1dfa-4c35-a5aa-0a020d60dc1d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:54.095993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557569d9-fcd7-4002-be93-ccead8a99361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:54.096637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557569d9-fcd7-4002-be93-ccead8a99361 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:54.096924 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557569d9-fcd7-4002-be93-ccead8a99361 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:54.097753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557569d9-fcd7-4002-be93-ccead8a99361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:54.113362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:54.195351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557569d9-fcd7-4002-be93-ccead8a99361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:54.196831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557569d9-fcd7-4002-be93-ccead8a99361 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:54.211509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f174c-62d8-48ee-b2bb-c4c143cf3350 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:54.211797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f174c-62d8-48ee-b2bb-c4c143cf3350 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:54.212339 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f174c-62d8-48ee-b2bb-c4c143cf3350 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:54.214047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 542/1082] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:01:54.221837 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7afbe7-3183-4cc8-ba10-6fcc80110c88 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:54.222716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7afbe7-3183-4cc8-ba10-6fcc80110c88 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:54.222972 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7afbe7-3183-4cc8-ba10-6fcc80110c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:54.243798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1e7afbe7-3183-4cc8-ba10-6fcc80110c88 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:54.245024 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1e7afbe7-3183-4cc8-ba10-6fcc80110c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:54.320622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 543/1083] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:54 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:54.326184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 541/1084] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:53 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 15:01:54.327751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7afbe7-3183-4cc8-ba10-6fcc80110c88 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:54.344248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557569d9-fcd7-4002-be93-ccead8a99361 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:54.397493 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:54.403925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:54.462329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557569d9-fcd7-4002-be93-ccead8a99361 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:54.464950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7afbe7-3183-4cc8-ba10-6fcc80110c88 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:54.507616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a5add-af69-48b4-945c-88b97e59775a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:54.508273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a5add-af69-48b4-945c-88b97e59775a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:54.508554 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a5add-af69-48b4-945c-88b97e59775a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:54.508829 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a5add-af69-48b4-945c-88b97e59775a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:54.509729 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f1fbc-c8fc-4383-a053-232d9d3444d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:54.510449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1fbc-c8fc-4383-a053-232d9d3444d7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:54.510845 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1fbc-c8fc-4383-a053-232d9d3444d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:54.511129 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f1fbc-c8fc-4383-a053-232d9d3444d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:54.520899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a5add-af69-48b4-945c-88b97e59775a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:54.522504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a5add-af69-48b4-945c-88b97e59775a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:54.523290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f1fbc-c8fc-4383-a053-232d9d3444d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-3f9f1fbc-c8fc-4383-a053-232d9d3444d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:54.528322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:54.529293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:54.530225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:54.530225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:54.530225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:54.530225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:54.530225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:54.530225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:54.530225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:54.530225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:54.530225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:54.530225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:54.530225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 15:01:54.530225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:54.535663 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 542/1085] 10.222.0.100 () {56 vars in 1125 bytes} [Mon Aug 1 15:01:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:54.536867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f1fbc-c8fc-4383-a053-232d9d3444d7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-e36a5add-af69-48b4-945c-88b97e59775a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:54.544031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:54.545053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:54.545986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:54.545986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:54.545986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:54.545986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:54.545986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 544/1086] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:54.551304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a5add-af69-48b4-945c-88b97e59775a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:54.669623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f1fbc-c8fc-4383-a053-232d9d3444d7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:54.672106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:54.675248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a5add-af69-48b4-945c-88b97e59775a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:54.685303 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:54.715884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b522b44-27ab-4154-8e2c-205db9767151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:54.716515 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b522b44-27ab-4154-8e2c-205db9767151 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:54.716800 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b522b44-27ab-4154-8e2c-205db9767151 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:54.717081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b522b44-27ab-4154-8e2c-205db9767151 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:54.717834 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b522b44-27ab-4154-8e2c-205db9767151 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:54.718141 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b522b44-27ab-4154-8e2c-205db9767151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:54.719552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b522b44-27ab-4154-8e2c-205db9767151 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:54.719668 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807829fd-8996-48e5-82a0-08e43f300f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:54.720275 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807829fd-8996-48e5-82a0-08e43f300f68 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:54.720623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807829fd-8996-48e5-82a0-08e43f300f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:54.720968 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807829fd-8996-48e5-82a0-08e43f300f68 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:54.721668 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807829fd-8996-48e5-82a0-08e43f300f68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:54.722170 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807829fd-8996-48e5-82a0-08e43f300f68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:54.723582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807829fd-8996-48e5-82a0-08e43f300f68 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:54.734066 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 543/1087] 10.222.0.100 () {56 vars in 1120 bytes} [Mon Aug 1 15:01:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:54.735058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b522b44-27ab-4154-8e2c-205db9767151 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:54.736666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 545/1088] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:54.867544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b522b44-27ab-4154-8e2c-205db9767151 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:54.911216 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6aa55b-b346-48aa-89f4-1adc8ab22de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:54.911881 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aa55b-b346-48aa-89f4-1adc8ab22de4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:54.912169 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aa55b-b346-48aa-89f4-1adc8ab22de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:54.912449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6aa55b-b346-48aa-89f4-1adc8ab22de4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:54.924816 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6aa55b-b346-48aa-89f4-1adc8ab22de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:54.926249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6aa55b-b346-48aa-89f4-1adc8ab22de4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-df6aa55b-b346-48aa-89f4-1adc8ab22de4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:54.944491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:54.945510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:54.946522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:54.946522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:54.946522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:54.946522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:54.946522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 544/1089] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:54.952992 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807829fd-8996-48e5-82a0-08e43f300f68 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:55.079547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807829fd-8996-48e5-82a0-08e43f300f68 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:55.131469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca50e9d-80fc-491a-b3de-ea736b949050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:55.132095 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca50e9d-80fc-491a-b3de-ea736b949050 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:55.132382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca50e9d-80fc-491a-b3de-ea736b949050 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:55.132756 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca50e9d-80fc-491a-b3de-ea736b949050 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:55.133465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca50e9d-80fc-491a-b3de-ea736b949050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:55.133776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca50e9d-80fc-491a-b3de-ea736b949050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:55.135211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca50e9d-80fc-491a-b3de-ea736b949050 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:55.147761 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 546/1090] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:55.153347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6aa55b-b346-48aa-89f4-1adc8ab22de4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:55.175783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:55.275104 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6aa55b-b346-48aa-89f4-1adc8ab22de4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:55.306226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94775fef-8c37-455b-9e64-d68022c7ddd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:55.306902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94775fef-8c37-455b-9e64-d68022c7ddd4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:55.307189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94775fef-8c37-455b-9e64-d68022c7ddd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:55.307467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94775fef-8c37-455b-9e64-d68022c7ddd4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:55.315382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94775fef-8c37-455b-9e64-d68022c7ddd4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:55.316861 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94775fef-8c37-455b-9e64-d68022c7ddd4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-94775fef-8c37-455b-9e64-d68022c7ddd4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:01:55.328523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:55.329749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:55.330763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:55.330763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:55.330763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:01:55.330763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:01:55.330763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:01:55.330763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:01:55.330763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:01:55.330763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:01:55.330763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 15:01:55.330763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:55.330763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 545/1091] 10.222.0.100 () {56 vars in 1098 bytes} [Mon Aug 1 15:01:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:55.335948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca50e9d-80fc-491a-b3de-ea736b949050 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:55.365395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:55.438272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca50e9d-80fc-491a-b3de-ea736b949050 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:55.473600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504245b-022e-4aa6-ab73-5a0f7e9cb4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:55.474227 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504245b-022e-4aa6-ab73-5a0f7e9cb4ac None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:55.474506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504245b-022e-4aa6-ab73-5a0f7e9cb4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:55.474813 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504245b-022e-4aa6-ab73-5a0f7e9cb4ac None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:55.475609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504245b-022e-4aa6-ab73-5a0f7e9cb4ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:55.475918 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504245b-022e-4aa6-ab73-5a0f7e9cb4ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:55.477355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504245b-022e-4aa6-ab73-5a0f7e9cb4ac None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:55.500846 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 547/1092] 10.222.0.100 () {56 vars in 1138 bytes} [Mon Aug 1 15:01:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:55.506878 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94775fef-8c37-455b-9e64-d68022c7ddd4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:55.590641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:55.614112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94775fef-8c37-455b-9e64-d68022c7ddd4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:55.656958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddefc5-7c13-4ba7-9732-f9016eaf0687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:55.657586 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddefc5-7c13-4ba7-9732-f9016eaf0687 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:55.657877 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddefc5-7c13-4ba7-9732-f9016eaf0687 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:55.658155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddefc5-7c13-4ba7-9732-f9016eaf0687 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:55.731953 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddefc5-7c13-4ba7-9732-f9016eaf0687 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:55.733335 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddefc5-7c13-4ba7-9732-f9016eaf0687 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:55.817751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:55.842689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 546/1093] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:55 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:01:55.848423 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504245b-022e-4aa6-ab73-5a0f7e9cb4ac None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:55.936529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:55.953132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504245b-022e-4aa6-ab73-5a0f7e9cb4ac None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:55.988910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89282ac-1cfe-4f43-8e26-a96c84620b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:55.989464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89282ac-1cfe-4f43-8e26-a96c84620b7e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:55.989743 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89282ac-1cfe-4f43-8e26-a96c84620b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:55.990019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89282ac-1cfe-4f43-8e26-a96c84620b7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:56.001036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89282ac-1cfe-4f43-8e26-a96c84620b7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:56.002593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89282ac-1cfe-4f43-8e26-a96c84620b7e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-d89282ac-1cfe-4f43-8e26-a96c84620b7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:01:56.018422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:56.019500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:56.020400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:01:56.020400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:01:56.020400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:01:56.020400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:56.020400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 548/1094] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:56.025374 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddefc5-7c13-4ba7-9732-f9016eaf0687 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:56.085625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73ff65-7b4b-4941-b51e-2c227f085ec0 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:56.086164 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73ff65-7b4b-4941-b51e-2c227f085ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:56.086164 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73ff65-7b4b-4941-b51e-2c227f085ec0 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:56.087820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 549/1095] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:01:56.093475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648dd98-e546-4139-9c80-cd7df696b2d5 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:56.093760 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648dd98-e546-4139-9c80-cd7df696b2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:56.094035 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648dd98-e546-4139-9c80-cd7df696b2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:56.115374 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b648dd98-e546-4139-9c80-cd7df696b2d5 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:56.116584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b648dd98-e546-4139-9c80-cd7df696b2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:56.134922 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddefc5-7c13-4ba7-9732-f9016eaf0687 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:56.170759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942f334-3d6d-4909-b91e-8fe31a3bb5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:56.171455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942f334-3d6d-4909-b91e-8fe31a3bb5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:56.171647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942f334-3d6d-4909-b91e-8fe31a3bb5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:56.171923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942f334-3d6d-4909-b91e-8fe31a3bb5f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:56.172624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942f334-3d6d-4909-b91e-8fe31a3bb5f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:56.172929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942f334-3d6d-4909-b91e-8fe31a3bb5f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:56.174314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942f334-3d6d-4909-b91e-8fe31a3bb5f1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:56.186965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 547/1096] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:01:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:56.187941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 550/1097] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:56 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:56.192518 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942f334-3d6d-4909-b91e-8fe31a3bb5f1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:56.210640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648dd98-e546-4139-9c80-cd7df696b2d5 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:56.228707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:56.282948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:56.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942f334-3d6d-4909-b91e-8fe31a3bb5f1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:56.320082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648dd98-e546-4139-9c80-cd7df696b2d5 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:56.339877 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839dfd3-c348-4d19-892f-dfb238b7aa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:56.340529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839dfd3-c348-4d19-892f-dfb238b7aa62 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:56.340811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839dfd3-c348-4d19-892f-dfb238b7aa62 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:56.341092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839dfd3-c348-4d19-892f-dfb238b7aa62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:56.348562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839dfd3-c348-4d19-892f-dfb238b7aa62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:56.350017 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839dfd3-c348-4d19-892f-dfb238b7aa62 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:56.360030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8c29b-6c4c-4f05-9e97-1f66ed9f4c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:56.360671 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8c29b-6c4c-4f05-9e97-1f66ed9f4c0b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:56.360950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8c29b-6c4c-4f05-9e97-1f66ed9f4c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:56.361221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8c29b-6c4c-4f05-9e97-1f66ed9f4c0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-5839dfd3-c348-4d19-892f-dfb238b7aa62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:01:56.361886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:56.363555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:56.364452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:01:56.364452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:01:56.364452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:01:56.364452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:01:56.364452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:01:56.364452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:01:56.364452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:01:56.364452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:01:56.364452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 548/1098] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:01:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:01:56.369455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839dfd3-c348-4d19-892f-dfb238b7aa62 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:56.375335 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8c29b-6c4c-4f05-9e97-1f66ed9f4c0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-cca8c29b-6c4c-4f05-9e97-1f66ed9f4c0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:01:56.380401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:01:56.382764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:01:56.383911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:01:56.383911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:01:56.383911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:01:56.383911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:01:56.383911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:01:56.383911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:01:56.383911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:01:56.383911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:01:56.383911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:01:56.383911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:01:56.383911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:01:56.383911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:01:56.383911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 551/1099] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:01:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:01:56.389377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8c29b-6c4c-4f05-9e97-1f66ed9f4c0b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:56.477616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:56.484121 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839dfd3-c348-4d19-892f-dfb238b7aa62 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:56.493425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8c29b-6c4c-4f05-9e97-1f66ed9f4c0b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:56.524082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:56.530367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aeb24-932c-4246-8220-6f8b357231e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:56.531276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aeb24-932c-4246-8220-6f8b357231e2 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:56.531784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aeb24-932c-4246-8220-6f8b357231e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:56.532245 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aeb24-932c-4246-8220-6f8b357231e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:56.533273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882aeb24-932c-4246-8220-6f8b357231e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:56.533579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882aeb24-932c-4246-8220-6f8b357231e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:56.533715 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1de2a-2e76-4e4c-abce-c70bcc940c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:56.534340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1de2a-2e76-4e4c-abce-c70bcc940c43 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:56.535206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1de2a-2e76-4e4c-abce-c70bcc940c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:56.535545 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1de2a-2e76-4e4c-abce-c70bcc940c43 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:56.535664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882aeb24-932c-4246-8220-6f8b357231e2 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:56.536267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1de2a-2e76-4e4c-abce-c70bcc940c43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:56.546297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1de2a-2e76-4e4c-abce-c70bcc940c43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:56.546297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1de2a-2e76-4e4c-abce-c70bcc940c43 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:56.553702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 552/1100] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:01:56 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:56.553886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 549/1101] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:01:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:56.555742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1de2a-2e76-4e4c-abce-c70bcc940c43 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:56.557027 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aeb24-932c-4246-8220-6f8b357231e2 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:56.662805 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aeb24-932c-4246-8220-6f8b357231e2 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:56.663603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1de2a-2e76-4e4c-abce-c70bcc940c43 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:56.698269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933014d-fbc9-4e6b-a756-621d2907f507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:56.698947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933014d-fbc9-4e6b-a756-621d2907f507 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:56.699222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933014d-fbc9-4e6b-a756-621d2907f507 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:56.699495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933014d-fbc9-4e6b-a756-621d2907f507 None admin] PATH_INFO: `/v3/users/fbc2d9a22c934d7598d887aa49393419` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:56.700667 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933014d-fbc9-4e6b-a756-621d2907f507 None admin] RBAC: Authorizing `identity:get_user(user_id=fbc2d9a22c934d7598d887aa49393419)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:56.702107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933014d-fbc9-4e6b-a756-621d2907f507 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:56.703284 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c6bf3-d673-4ebb-8559-799ebe9e9bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:56.703767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 553/1102] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:01:56 2022] GET /identity/v3/users/fbc2d9a22c934d7598d887aa49393419 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:56.703919 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c6bf3-d673-4ebb-8559-799ebe9e9bbe None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:56.704208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c6bf3-d673-4ebb-8559-799ebe9e9bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:56.704490 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c6bf3-d673-4ebb-8559-799ebe9e9bbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:56.705292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c6bf3-d673-4ebb-8559-799ebe9e9bbe None admin] RBAC: Adding query filter params (role.id=46f67f665d95448798ec8aef267e78fb, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=ec50ac7c54324884a1b2a6c63d617907) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:56.705607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c6bf3-d673-4ebb-8559-799ebe9e9bbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:56.707670 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c6bf3-d673-4ebb-8559-799ebe9e9bbe None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:56.710074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933014d-fbc9-4e6b-a756-621d2907f507 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:56.714696 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 550/1103] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:56 2022] GET /identity/v3/role_assignments?user.id=ec50ac7c54324884a1b2a6c63d617907&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=46f67f665d95448798ec8aef267e78fb => generated 610 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:01:56.776128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:56.830547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933014d-fbc9-4e6b-a756-621d2907f507 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:56.878377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c6d4ba-8b6f-48f4-acc6-df1a4e6cdeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:56.880428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6d4ba-8b6f-48f4-acc6-df1a4e6cdeed None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:56.880718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6d4ba-8b6f-48f4-acc6-df1a4e6cdeed None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:56.881004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6d4ba-8b6f-48f4-acc6-df1a4e6cdeed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:56.961527 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c6d4ba-8b6f-48f4-acc6-df1a4e6cdeed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:56.963328 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c6d4ba-8b6f-48f4-acc6-df1a4e6cdeed None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:57.026554 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:57.084533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 554/1104] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:01:56 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 15:01:57.090716 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c6bf3-d673-4ebb-8559-799ebe9e9bbe None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:57.116676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:57.207856 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c6bf3-d673-4ebb-8559-799ebe9e9bbe None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:57.244385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d2508f-39cf-4be2-8085-b3b86a02c108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:57.245048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2508f-39cf-4be2-8085-b3b86a02c108 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:57.245352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2508f-39cf-4be2-8085-b3b86a02c108 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:57.245613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d2508f-39cf-4be2-8085-b3b86a02c108 None admin] PATH_INFO: `/v3/projects/1454aaf365d148c3a1cc9caec965e79e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:57.248320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2508f-39cf-4be2-8085-b3b86a02c108 None admin] RBAC: Authorizing `identity:get_project(project_id=1454aaf365d148c3a1cc9caec965e79e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:57.248320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d2508f-39cf-4be2-8085-b3b86a02c108 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:57.250416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 551/1105] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:01:57 2022] GET /identity/v3/projects/1454aaf365d148c3a1cc9caec965e79e => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:57.257304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c6d4ba-8b6f-48f4-acc6-df1a4e6cdeed None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:57.369762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c6d4ba-8b6f-48f4-acc6-df1a4e6cdeed None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:57.406756 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2a0c4-a0b7-45e7-a82f-96eb7e9c34b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:57.407389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2a0c4-a0b7-45e7-a82f-96eb7e9c34b7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:57.407667 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2a0c4-a0b7-45e7-a82f-96eb7e9c34b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:57.407935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2a0c4-a0b7-45e7-a82f-96eb7e9c34b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:57.408733 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2a0c4-a0b7-45e7-a82f-96eb7e9c34b7 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=1454aaf365d148c3a1cc9caec965e79e, user.id=fbc2d9a22c934d7598d887aa49393419) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:01:57.409044 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2a0c4-a0b7-45e7-a82f-96eb7e9c34b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:57.410521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2a0c4-a0b7-45e7-a82f-96eb7e9c34b7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:57.418355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 555/1106] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:01:57 2022] GET /identity/v3/role_assignments?user.id=fbc2d9a22c934d7598d887aa49393419&scope.project.id=1454aaf365d148c3a1cc9caec965e79e&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 610 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:59.501499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585b657-eee3-4555-8618-6d861103043d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:59.502041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585b657-eee3-4555-8618-6d861103043d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:59.502041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585b657-eee3-4555-8618-6d861103043d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:59.504013 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 552/1107] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:01:59 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:01:59.509257 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f61fd9-d135-409d-a532-035fbde3f605 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:59.509692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f61fd9-d135-409d-a532-035fbde3f605 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:59.509808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f61fd9-d135-409d-a532-035fbde3f605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:59.527472 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-92f61fd9-d135-409d-a532-035fbde3f605 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:01:59.528652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-92f61fd9-d135-409d-a532-035fbde3f605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:01:59.546390 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:59.598153 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 556/1108] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:01:59 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:01:59.623171 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585b657-eee3-4555-8618-6d861103043d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:59.713277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:01:59.727604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585b657-eee3-4555-8618-6d861103043d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:59.761331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfeb63-51c1-4afb-b82b-10589ea48a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:59.761894 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfeb63-51c1-4afb-b82b-10589ea48a6b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:59.762176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfeb63-51c1-4afb-b82b-10589ea48a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:59.762452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfeb63-51c1-4afb-b82b-10589ea48a6b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:59.763644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfeb63-51c1-4afb-b82b-10589ea48a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:59.765084 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfeb63-51c1-4afb-b82b-10589ea48a6b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:59.766722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 553/1109] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:01:59 2022] GET /identity/v3/domains/default => generated 200 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:01:59.772745 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f61fd9-d135-409d-a532-035fbde3f605 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:01:59.889046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f61fd9-d135-409d-a532-035fbde3f605 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:01:59.925992 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b9285-2853-42d0-9b37-597191234a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:01:59.926610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b9285-2853-42d0-9b37-597191234a46 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:01:59.926916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b9285-2853-42d0-9b37-597191234a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:01:59.927188 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b9285-2853-42d0-9b37-597191234a46 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:01:59.927874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b9285-2853-42d0-9b37-597191234a46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:01:59.929318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b9285-2853-42d0-9b37-597191234a46 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:01:59.951368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 557/1110] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:01:59 2022] POST /identity/v3/users => generated 287 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:02:02.024388 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c6387-cebb-4d05-8754-7943405ba771 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:02.024958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c6387-cebb-4d05-8754-7943405ba771 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:02.024958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3c6387-cebb-4d05-8754-7943405ba771 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:02.026283 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 554/1111] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:02.031689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb3412-7662-464c-9d16-984372751252 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:02.032345 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb3412-7662-464c-9d16-984372751252 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:02.032345 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb3412-7662-464c-9d16-984372751252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:02.051177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-28fb3412-7662-464c-9d16-984372751252 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:02.052392 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-28fb3412-7662-464c-9d16-984372751252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:02.054452 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:02.126768 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 558/1112] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:02 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:02.150451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c6387-cebb-4d05-8754-7943405ba771 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:02.251364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3c6387-cebb-4d05-8754-7943405ba771 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:02.285880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f7862-13b9-48d6-afc7-d683f7d4c4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:02.286462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f7862-13b9-48d6-afc7-d683f7d4c4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:02.286773 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f7862-13b9-48d6-afc7-d683f7d4c4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:02.287056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f7862-13b9-48d6-afc7-d683f7d4c4a4 None admin] PATH_INFO: `/v3/roles/57221d717a454c7fae4f2df1de9dbe5b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:02.288502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f7862-13b9-48d6-afc7-d683f7d4c4a4 None admin] RBAC: Authorizing `identity:get_role(role_id=57221d717a454c7fae4f2df1de9dbe5b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:02.289874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f7862-13b9-48d6-afc7-d683f7d4c4a4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:02.291267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 555/1113] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:02:02 2022] GET /identity/v3/roles/57221d717a454c7fae4f2df1de9dbe5b => generated 224 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:02.297197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb3412-7662-464c-9d16-984372751252 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:02.395149 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb3412-7662-464c-9d16-984372751252 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:02.427996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82783668-2c06-48a4-a2d5-f4ce2cdb4fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:02.428631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82783668-2c06-48a4-a2d5-f4ce2cdb4fbc None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:02.428910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82783668-2c06-48a4-a2d5-f4ce2cdb4fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:02.429181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82783668-2c06-48a4-a2d5-f4ce2cdb4fbc None admin] PATH_INFO: `/v3/users/ce1a9c83f7c74532b2206f979090c99a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:02.436508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:02.443638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82783668-2c06-48a4-a2d5-f4ce2cdb4fbc None admin] RBAC: Authorizing `identity:get_user(user_id=ce1a9c83f7c74532b2206f979090c99a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:02.445087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82783668-2c06-48a4-a2d5-f4ce2cdb4fbc None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:02.446802 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 559/1114] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:02:02 2022] GET /identity/v3/users/ce1a9c83f7c74532b2206f979090c99a => generated 287 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:02.452635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f7862-13b9-48d6-afc7-d683f7d4c4a4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:02.481400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:02.559764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f7862-13b9-48d6-afc7-d683f7d4c4a4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:02.594089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8a9ed-13d9-4f8b-94e9-735d9ba0b587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:02.594753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8a9ed-13d9-4f8b-94e9-735d9ba0b587 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:02.595040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8a9ed-13d9-4f8b-94e9-735d9ba0b587 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:02.595318 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8a9ed-13d9-4f8b-94e9-735d9ba0b587 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:02.663487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8a9ed-13d9-4f8b-94e9-735d9ba0b587 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:02.664976 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8a9ed-13d9-4f8b-94e9-735d9ba0b587 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:02.691259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:02.768788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 556/1115] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:02:02 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:02:02.774709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82783668-2c06-48a4-a2d5-f4ce2cdb4fbc None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:02.895530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82783668-2c06-48a4-a2d5-f4ce2cdb4fbc None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:02.931045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee988a-e92c-4666-83ff-19f632fcaaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:02.931694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee988a-e92c-4666-83ff-19f632fcaaca None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:02.931986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee988a-e92c-4666-83ff-19f632fcaaca None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:02.932265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee988a-e92c-4666-83ff-19f632fcaaca None admin] PATH_INFO: `/v3/projects/1454aaf365d148c3a1cc9caec965e79e` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:02.933432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee988a-e92c-4666-83ff-19f632fcaaca None admin] RBAC: Authorizing `identity:get_project(project_id=1454aaf365d148c3a1cc9caec965e79e)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:02.934910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee988a-e92c-4666-83ff-19f632fcaaca None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:02.936623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 560/1116] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:02:02 2022] GET /identity/v3/projects/1454aaf365d148c3a1cc9caec965e79e => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:02.942806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8a9ed-13d9-4f8b-94e9-735d9ba0b587 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:03.046521 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8a9ed-13d9-4f8b-94e9-735d9ba0b587 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:03.070843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:03.082020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bcb688-2a03-4b2d-ba9d-c589f4068037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:03.082646 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcb688-2a03-4b2d-ba9d-c589f4068037 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:03.082970 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcb688-2a03-4b2d-ba9d-c589f4068037 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:03.083246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcb688-2a03-4b2d-ba9d-c589f4068037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:03.084040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bcb688-2a03-4b2d-ba9d-c589f4068037 None admin] RBAC: Adding query filter params (role.id=57221d717a454c7fae4f2df1de9dbe5b, scope.project.id=1454aaf365d148c3a1cc9caec965e79e, user.id=ce1a9c83f7c74532b2206f979090c99a) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:03.084354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bcb688-2a03-4b2d-ba9d-c589f4068037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:03.085801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bcb688-2a03-4b2d-ba9d-c589f4068037 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:03.091763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 557/1117] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:02:02 2022] GET /identity/v3/role_assignments?user.id=ce1a9c83f7c74532b2206f979090c99a&scope.project.id=1454aaf365d148c3a1cc9caec965e79e&role.id=57221d717a454c7fae4f2df1de9dbe5b => generated 260 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:05.092022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70f10b-7932-4d48-9cad-3158323f937e None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:05.092563 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70f10b-7932-4d48-9cad-3158323f937e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:05.092563 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70f10b-7932-4d48-9cad-3158323f937e None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:05.093939 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 561/1118] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:05.099964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e4e9e-fc33-48e4-9c85-171470eceb9b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:05.100400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e4e9e-fc33-48e4-9c85-171470eceb9b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:05.100519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e4e9e-fc33-48e4-9c85-171470eceb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:05.121752 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-347e4e9e-fc33-48e4-9c85-171470eceb9b None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:05.121752 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-347e4e9e-fc33-48e4-9c85-171470eceb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:05.192310 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 558/1119] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:05 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:05.215235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70f10b-7932-4d48-9cad-3158323f937e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:05.283249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:05.322362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70f10b-7932-4d48-9cad-3158323f937e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:05.357021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e5e9b-d7b3-4422-b478-f1b8206109cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:05.357577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e5e9b-d7b3-4422-b478-f1b8206109cb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:05.357856 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e5e9b-d7b3-4422-b478-f1b8206109cb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:05.358132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e5e9b-d7b3-4422-b478-f1b8206109cb None admin] PATH_INFO: `/v3/roles/57221d717a454c7fae4f2df1de9dbe5b` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:05.359616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e5e9b-d7b3-4422-b478-f1b8206109cb None admin] RBAC: Authorizing `identity:get_role(role_id=57221d717a454c7fae4f2df1de9dbe5b)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:05.360995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e5e9b-d7b3-4422-b478-f1b8206109cb None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:05.362391 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 562/1120] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:02:05 2022] GET /identity/v3/roles/57221d717a454c7fae4f2df1de9dbe5b => generated 224 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:05.368370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e4e9e-fc33-48e4-9c85-171470eceb9b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:05.469299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e4e9e-fc33-48e4-9c85-171470eceb9b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:05.486909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:05.504528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27235e58-f562-466d-a6cd-9d081a368b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:05.505180 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27235e58-f562-466d-a6cd-9d081a368b33 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:05.505464 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27235e58-f562-466d-a6cd-9d081a368b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:05.505742 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27235e58-f562-466d-a6cd-9d081a368b33 None admin] PATH_INFO: `/v3/users/ce1a9c83f7c74532b2206f979090c99a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:05.506944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27235e58-f562-466d-a6cd-9d081a368b33 None admin] RBAC: Authorizing `identity:get_user(user_id=ce1a9c83f7c74532b2206f979090c99a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:05.508425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27235e58-f562-466d-a6cd-9d081a368b33 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:05.510049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 559/1121] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:02:05 2022] GET /identity/v3/users/ce1a9c83f7c74532b2206f979090c99a => generated 287 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:05.515983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e5e9b-d7b3-4422-b478-f1b8206109cb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:05.623320 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e5e9b-d7b3-4422-b478-f1b8206109cb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:05.653383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:05.658870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d01cd-c80a-45d2-a320-b0fddfc7cd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:05.659525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d01cd-c80a-45d2-a320-b0fddfc7cd0f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:05.659811 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d01cd-c80a-45d2-a320-b0fddfc7cd0f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:05.660091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d01cd-c80a-45d2-a320-b0fddfc7cd0f None admin] PATH_INFO: `/v3/projects/1454aaf365d148c3a1cc9caec965e79e` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:05.661255 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d01cd-c80a-45d2-a320-b0fddfc7cd0f None admin] RBAC: Authorizing `identity:get_project(project_id=1454aaf365d148c3a1cc9caec965e79e)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:05.662725 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d01cd-c80a-45d2-a320-b0fddfc7cd0f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:05.664424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 563/1122] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:02:05 2022] GET /identity/v3/projects/1454aaf365d148c3a1cc9caec965e79e => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:05.670419 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27235e58-f562-466d-a6cd-9d081a368b33 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:05.772453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27235e58-f562-466d-a6cd-9d081a368b33 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:05.808836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6522cab-27a2-44b2-b42e-6cf0aba1c8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:05.809508 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6522cab-27a2-44b2-b42e-6cf0aba1c8f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:05.811261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6522cab-27a2-44b2-b42e-6cf0aba1c8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:05.811261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6522cab-27a2-44b2-b42e-6cf0aba1c8f1 None admin] PATH_INFO: `/v3/projects/1454aaf365d148c3a1cc9caec965e79e/users/ce1a9c83f7c74532b2206f979090c99a/roles/57221d717a454c7fae4f2df1de9dbe5b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:05.812328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6522cab-27a2-44b2-b42e-6cf0aba1c8f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1454aaf365d148c3a1cc9caec965e79e, user_id=ce1a9c83f7c74532b2206f979090c99a, role_id=57221d717a454c7fae4f2df1de9dbe5b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:05.813772 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6522cab-27a2-44b2-b42e-6cf0aba1c8f1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:05.828416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 560/1123] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:02:05 2022] PUT /identity/v3/projects/1454aaf365d148c3a1cc9caec965e79e/users/ce1a9c83f7c74532b2206f979090c99a/roles/57221d717a454c7fae4f2df1de9dbe5b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:02:07.841417 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e2fcc-a1c0-46a6-bbdc-100513e41945 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:07.841947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e2fcc-a1c0-46a6-bbdc-100513e41945 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:07.841947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e2fcc-a1c0-46a6-bbdc-100513e41945 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:07.843291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 564/1124] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:07.848451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c8961-0e39-4aeb-89ba-ea0da3532e9a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:07.848886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c8961-0e39-4aeb-89ba-ea0da3532e9a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:07.849012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c8961-0e39-4aeb-89ba-ea0da3532e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:07.867102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-093c8961-0e39-4aeb-89ba-ea0da3532e9a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:07.868314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-093c8961-0e39-4aeb-89ba-ea0da3532e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:07.913644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:07.945837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 561/1125] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:07 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:07.969643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e2fcc-a1c0-46a6-bbdc-100513e41945 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:08.078145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e2fcc-a1c0-46a6-bbdc-100513e41945 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:08.114185 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb974b-5490-4652-97ba-07edf294e294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:08.115041 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb974b-5490-4652-97ba-07edf294e294 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:08.115329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb974b-5490-4652-97ba-07edf294e294 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:08.115606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb974b-5490-4652-97ba-07edf294e294 None admin] PATH_INFO: `/v3/roles/57221d717a454c7fae4f2df1de9dbe5b` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:08.117064 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb974b-5490-4652-97ba-07edf294e294 None admin] RBAC: Authorizing `identity:get_role(role_id=57221d717a454c7fae4f2df1de9dbe5b)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:08.118442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb974b-5490-4652-97ba-07edf294e294 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:08.119834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 565/1126] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:02:07 2022] GET /identity/v3/roles/57221d717a454c7fae4f2df1de9dbe5b => generated 224 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:08.125847 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c8961-0e39-4aeb-89ba-ea0da3532e9a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:08.227696 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c8961-0e39-4aeb-89ba-ea0da3532e9a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:08.262513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06636f-598d-46f0-9047-2e56a6a7bc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:08.263192 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06636f-598d-46f0-9047-2e56a6a7bc76 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:08.263475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06636f-598d-46f0-9047-2e56a6a7bc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:08.263751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06636f-598d-46f0-9047-2e56a6a7bc76 None admin] PATH_INFO: `/v3/users/ce1a9c83f7c74532b2206f979090c99a` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:08.266616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06636f-598d-46f0-9047-2e56a6a7bc76 None admin] RBAC: Authorizing `identity:get_user(user_id=ce1a9c83f7c74532b2206f979090c99a)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:08.268094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06636f-598d-46f0-9047-2e56a6a7bc76 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:08.269741 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 562/1127] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:02:08 2022] GET /identity/v3/users/ce1a9c83f7c74532b2206f979090c99a => generated 287 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:08.275782 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb974b-5490-4652-97ba-07edf294e294 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:08.342275 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:08.376384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb974b-5490-4652-97ba-07edf294e294 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:08.410252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3372664-6d1e-4e92-b4c8-0fe78f47561f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:08.410893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3372664-6d1e-4e92-b4c8-0fe78f47561f None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:08.411176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3372664-6d1e-4e92-b4c8-0fe78f47561f None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:08.411452 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3372664-6d1e-4e92-b4c8-0fe78f47561f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:08.479963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3372664-6d1e-4e92-b4c8-0fe78f47561f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:08.481423 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3372664-6d1e-4e92-b4c8-0fe78f47561f None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:08.547773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:08.583346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 566/1128] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:02:08 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:02:08.589700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06636f-598d-46f0-9047-2e56a6a7bc76 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:08.637029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:08.691792 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06636f-598d-46f0-9047-2e56a6a7bc76 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:08.726321 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2174ec7-63ae-44a7-a9ed-51839214d0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:08.726991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2174ec7-63ae-44a7-a9ed-51839214d0ac None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:08.727277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2174ec7-63ae-44a7-a9ed-51839214d0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:08.727557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2174ec7-63ae-44a7-a9ed-51839214d0ac None admin] PATH_INFO: `/v3/projects/1454aaf365d148c3a1cc9caec965e79e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:08.728721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2174ec7-63ae-44a7-a9ed-51839214d0ac None admin] RBAC: Authorizing `identity:get_project(project_id=1454aaf365d148c3a1cc9caec965e79e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:08.730164 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2174ec7-63ae-44a7-a9ed-51839214d0ac None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:08.731886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 563/1129] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:02:08 2022] GET /identity/v3/projects/1454aaf365d148c3a1cc9caec965e79e => generated 312 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:08.738633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3372664-6d1e-4e92-b4c8-0fe78f47561f None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:08.838301 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3372664-6d1e-4e92-b4c8-0fe78f47561f None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:08.873328 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e051ff0-95da-4062-b4c8-335aad639c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:08.874042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e051ff0-95da-4062-b4c8-335aad639c3d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:08.874217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e051ff0-95da-4062-b4c8-335aad639c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:08.874494 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e051ff0-95da-4062-b4c8-335aad639c3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:08.875320 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e051ff0-95da-4062-b4c8-335aad639c3d None admin] RBAC: Adding query filter params (role.id=57221d717a454c7fae4f2df1de9dbe5b, scope.project.id=1454aaf365d148c3a1cc9caec965e79e, user.id=ce1a9c83f7c74532b2206f979090c99a) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:08.875637 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e051ff0-95da-4062-b4c8-335aad639c3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:08.877080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e051ff0-95da-4062-b4c8-335aad639c3d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:08.883371 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 567/1130] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:02:08 2022] GET /identity/v3/role_assignments?user.id=ce1a9c83f7c74532b2206f979090c99a&scope.project.id=1454aaf365d148c3a1cc9caec965e79e&role.id=57221d717a454c7fae4f2df1de9dbe5b => generated 610 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:10.899699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edc6c2-6ac4-4583-a8f8-494fbbc06d82 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:10.900269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edc6c2-6ac4-4583-a8f8-494fbbc06d82 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:10.900269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edc6c2-6ac4-4583-a8f8-494fbbc06d82 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:10.901680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 564/1131] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:10.907254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91cb79-b85e-464c-9e45-4080e9c15207 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:10.907742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91cb79-b85e-464c-9e45-4080e9c15207 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:10.907859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91cb79-b85e-464c-9e45-4080e9c15207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:10.925886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2d91cb79-b85e-464c-9e45-4080e9c15207 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:10.927135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2d91cb79-b85e-464c-9e45-4080e9c15207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:10.961138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:11.006143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 568/1132] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:10 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:11.029767 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edc6c2-6ac4-4583-a8f8-494fbbc06d82 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:11.140682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edc6c2-6ac4-4583-a8f8-494fbbc06d82 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:11.154647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:11.176477 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553079b-4658-4af1-8f69-bb771fd3666b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:11.177110 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553079b-4658-4af1-8f69-bb771fd3666b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:11.177388 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553079b-4658-4af1-8f69-bb771fd3666b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:11.177661 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553079b-4658-4af1-8f69-bb771fd3666b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:11.178849 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553079b-4658-4af1-8f69-bb771fd3666b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:11.180281 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553079b-4658-4af1-8f69-bb771fd3666b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:11.181874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 565/1133] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:02:11 2022] GET /identity/v3/domains/default => generated 200 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:11.187860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91cb79-b85e-464c-9e45-4080e9c15207 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:11.294545 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91cb79-b85e-464c-9e45-4080e9c15207 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:11.329744 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20cbb65-fdbd-4e8f-b628-4de87c18030a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:11.330376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cbb65-fdbd-4e8f-b628-4de87c18030a None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:11.330721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cbb65-fdbd-4e8f-b628-4de87c18030a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:11.330965 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cbb65-fdbd-4e8f-b628-4de87c18030a None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:11.331662 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20cbb65-fdbd-4e8f-b628-4de87c18030a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:11.333110 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20cbb65-fdbd-4e8f-b628-4de87c18030a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:11.362315 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 569/1134] 10.222.0.100 () {60 vars in 1146 bytes} [Mon Aug 1 15:02:11 2022] POST /identity/v3/projects => generated 312 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:02:13.457751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74d694-ac74-469d-9b7a-e44bdedfa12f None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:13.458364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74d694-ac74-469d-9b7a-e44bdedfa12f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:13.458364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74d694-ac74-469d-9b7a-e44bdedfa12f None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:13.459705 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 566/1135] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:13.465238 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d038d-8dd3-42cc-941c-aba1c0e952e9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:13.465703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d038d-8dd3-42cc-941c-aba1c0e952e9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:13.465815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d038d-8dd3-42cc-941c-aba1c0e952e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:13.485484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c83d038d-8dd3-42cc-941c-aba1c0e952e9 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:13.486706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c83d038d-8dd3-42cc-941c-aba1c0e952e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:13.496632 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:13.557459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 570/1136] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:13 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:13.582175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74d694-ac74-469d-9b7a-e44bdedfa12f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:13.685032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74d694-ac74-469d-9b7a-e44bdedfa12f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:13.719369 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1626c-9475-4e5b-ba35-4c247ae0ad68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:13.719932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1626c-9475-4e5b-ba35-4c247ae0ad68 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:13.720217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1626c-9475-4e5b-ba35-4c247ae0ad68 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:13.720494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1626c-9475-4e5b-ba35-4c247ae0ad68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:13.722263 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1626c-9475-4e5b-ba35-4c247ae0ad68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:13.723743 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1626c-9475-4e5b-ba35-4c247ae0ad68 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:13.725350 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 567/1137] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:02:13 2022] GET /identity/v3/domains/default => generated 200 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:13.731473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d038d-8dd3-42cc-941c-aba1c0e952e9 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:13.831308 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d038d-8dd3-42cc-941c-aba1c0e952e9 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:13.865524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46516c2-a9b4-4fb0-8c7a-72df889a4087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:13.866143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46516c2-a9b4-4fb0-8c7a-72df889a4087 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:13.866424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46516c2-a9b4-4fb0-8c7a-72df889a4087 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:13.866718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46516c2-a9b4-4fb0-8c7a-72df889a4087 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:13.867405 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46516c2-a9b4-4fb0-8c7a-72df889a4087 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:13.868850 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46516c2-a9b4-4fb0-8c7a-72df889a4087 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:13.886885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:13.892811 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 571/1138] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:02:13 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:02:15.952275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e2bf1-69e9-478a-8813-3b7621614cc5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:15.952977 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e2bf1-69e9-478a-8813-3b7621614cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:15.952977 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e2bf1-69e9-478a-8813-3b7621614cc5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:15.954187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 568/1139] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:15.960815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbb773-b1b9-48f0-ab2b-46598bc08c99 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:15.961279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbb773-b1b9-48f0-ab2b-46598bc08c99 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:15.961381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fbb773-b1b9-48f0-ab2b-46598bc08c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:15.979870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c4fbb773-b1b9-48f0-ab2b-46598bc08c99 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:15.981078 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c4fbb773-b1b9-48f0-ab2b-46598bc08c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:16.052252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 572/1140] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:15 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:16.075638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e2bf1-69e9-478a-8813-3b7621614cc5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:16.127853 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:16.182253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e2bf1-69e9-478a-8813-3b7621614cc5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:16.216638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d47661-f673-4a5d-8c67-60d19d1e9fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:16.217195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d47661-f673-4a5d-8c67-60d19d1e9fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:16.217482 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d47661-f673-4a5d-8c67-60d19d1e9fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:16.217765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d47661-f673-4a5d-8c67-60d19d1e9fc1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:16.228938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d47661-f673-4a5d-8c67-60d19d1e9fc1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-c3d47661-f673-4a5d-8c67-60d19d1e9fc1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:02:16.233879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:02:16.235149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:02:16.236262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:02:16.236262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:02:16.236262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:02:16.236262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:02:16.236262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:02:16.236262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:02:16.236262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:02:16.236262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:02:16.236262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:02:16.236262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:02:16.236262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:02:16.236262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:02:16.236262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 569/1141] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:02:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:02:16.241748 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbb773-b1b9-48f0-ab2b-46598bc08c99 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:16.342503 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fbb773-b1b9-48f0-ab2b-46598bc08c99 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:16.372217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:16.378819 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c5ec8-63bc-43aa-8f42-0cab8c62ace5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:16.379454 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c5ec8-63bc-43aa-8f42-0cab8c62ace5 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:16.379740 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c5ec8-63bc-43aa-8f42-0cab8c62ace5 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:16.380018 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c5ec8-63bc-43aa-8f42-0cab8c62ace5 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:16.380747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c5ec8-63bc-43aa-8f42-0cab8c62ace5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:16.381056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c5ec8-63bc-43aa-8f42-0cab8c62ace5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:16.382437 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c5ec8-63bc-43aa-8f42-0cab8c62ace5 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:16.391155 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 573/1142] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:02:16 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:16.396990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d47661-f673-4a5d-8c67-60d19d1e9fc1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:16.501737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d47661-f673-4a5d-8c67-60d19d1e9fc1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:16.511096 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:16.538187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b509911-5c71-48ba-8fa7-f3f4feb3f53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:16.538881 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b509911-5c71-48ba-8fa7-f3f4feb3f53c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:16.539169 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b509911-5c71-48ba-8fa7-f3f4feb3f53c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:16.539450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b509911-5c71-48ba-8fa7-f3f4feb3f53c None admin] PATH_INFO: `/v3/users/6f69eaa447474f11a3193c5a5b8dcd25` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:16.555262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b509911-5c71-48ba-8fa7-f3f4feb3f53c None admin] RBAC: Authorizing `identity:get_user(user_id=6f69eaa447474f11a3193c5a5b8dcd25)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:16.557538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b509911-5c71-48ba-8fa7-f3f4feb3f53c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:16.559324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 570/1143] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:02:16 2022] GET /identity/v3/users/6f69eaa447474f11a3193c5a5b8dcd25 => generated 287 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:16.565116 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c5ec8-63bc-43aa-8f42-0cab8c62ace5 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:16.677493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c5ec8-63bc-43aa-8f42-0cab8c62ace5 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:16.716800 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a4b00-b08a-4b07-b681-be14969cefa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:16.717424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a4b00-b08a-4b07-b681-be14969cefa0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:16.717708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a4b00-b08a-4b07-b681-be14969cefa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:16.717984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a4b00-b08a-4b07-b681-be14969cefa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:16.778541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:16.789604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a4b00-b08a-4b07-b681-be14969cefa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:16.791109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4a4b00-b08a-4b07-b681-be14969cefa0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:16.891336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 574/1144] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:02:16 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:02:16.897881 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b509911-5c71-48ba-8fa7-f3f4feb3f53c None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:16.998790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b509911-5c71-48ba-8fa7-f3f4feb3f53c None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:17.033147 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01469465-df19-42d4-a3d6-85452b0e0ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:17.033739 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01469465-df19-42d4-a3d6-85452b0e0ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:17.034020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01469465-df19-42d4-a3d6-85452b0e0ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:17.034295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01469465-df19-42d4-a3d6-85452b0e0ab6 None admin] PATH_INFO: `/v3/projects/92df8a206218448c9b0ab8f40e3196b4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:17.035595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01469465-df19-42d4-a3d6-85452b0e0ab6 None admin] RBAC: Authorizing `identity:get_project(project_id=92df8a206218448c9b0ab8f40e3196b4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:17.037752 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01469465-df19-42d4-a3d6-85452b0e0ab6 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:17.040457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 571/1145] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:02:16 2022] GET /identity/v3/projects/92df8a206218448c9b0ab8f40e3196b4 => generated 312 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:17.046925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a4b00-b08a-4b07-b681-be14969cefa0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:17.137958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:17.147948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a4b00-b08a-4b07-b681-be14969cefa0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:17.181979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b433a-7b4f-4b56-a53c-1ff1467bbb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:17.182636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b433a-7b4f-4b56-a53c-1ff1467bbb42 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:17.182952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b433a-7b4f-4b56-a53c-1ff1467bbb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:17.183230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b433a-7b4f-4b56-a53c-1ff1467bbb42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:17.184026 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b433a-7b4f-4b56-a53c-1ff1467bbb42 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=92df8a206218448c9b0ab8f40e3196b4, user.id=6f69eaa447474f11a3193c5a5b8dcd25) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:17.184340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b433a-7b4f-4b56-a53c-1ff1467bbb42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:17.185778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b433a-7b4f-4b56-a53c-1ff1467bbb42 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:17.192095 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 575/1146] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:02:17 2022] GET /identity/v3/role_assignments?user.id=6f69eaa447474f11a3193c5a5b8dcd25&scope.project.id=92df8a206218448c9b0ab8f40e3196b4&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 260 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:19.210351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475e813-6d77-47b0-922b-e2af95546be0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:19.210946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475e813-6d77-47b0-922b-e2af95546be0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:19.210946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475e813-6d77-47b0-922b-e2af95546be0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:19.212543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 572/1147] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:19.219373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca12bd0-c32c-4275-80bc-0623b0ea173e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:19.219882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca12bd0-c32c-4275-80bc-0623b0ea173e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:19.220311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca12bd0-c32c-4275-80bc-0623b0ea173e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:19.239960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2ca12bd0-c32c-4275-80bc-0623b0ea173e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:19.241156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2ca12bd0-c32c-4275-80bc-0623b0ea173e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:19.318047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 576/1148] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:19 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:19.345459 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475e813-6d77-47b0-922b-e2af95546be0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:19.382579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:19.455902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475e813-6d77-47b0-922b-e2af95546be0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:19.490005 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400c287-afdb-4c22-88da-27ee1c686487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:19.490561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c287-afdb-4c22-88da-27ee1c686487 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:19.490867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c287-afdb-4c22-88da-27ee1c686487 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:19.491140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c287-afdb-4c22-88da-27ee1c686487 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:19.501801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400c287-afdb-4c22-88da-27ee1c686487 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a400c287-afdb-4c22-88da-27ee1c686487 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:02:19.506825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:02:19.507990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:02:19.509309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:02:19.509309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:02:19.509309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:02:19.509309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:02:19.509309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:02:19.509309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:02:19.509309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:02:19.509309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:02:19.509309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:02:19.509309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:02:19.509309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:02:19.509309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:02:19.509309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 573/1149] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:02:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:02:19.514232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca12bd0-c32c-4275-80bc-0623b0ea173e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:19.604922 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:19.620477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca12bd0-c32c-4275-80bc-0623b0ea173e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:19.655596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f2a29-27d2-4b5c-a99c-82340d705a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:19.656229 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f2a29-27d2-4b5c-a99c-82340d705a47 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:19.656514 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f2a29-27d2-4b5c-a99c-82340d705a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:19.656798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f2a29-27d2-4b5c-a99c-82340d705a47 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:19.657518 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f2a29-27d2-4b5c-a99c-82340d705a47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:19.657827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f2a29-27d2-4b5c-a99c-82340d705a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:19.659853 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f2a29-27d2-4b5c-a99c-82340d705a47 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:19.668449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 577/1150] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:02:19 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:19.674839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400c287-afdb-4c22-88da-27ee1c686487 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:19.770875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:19.778076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400c287-afdb-4c22-88da-27ee1c686487 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:19.812870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129732c4-9b23-4a2b-a731-91f9e5814b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:19.813519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129732c4-9b23-4a2b-a731-91f9e5814b38 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:19.813805 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129732c4-9b23-4a2b-a731-91f9e5814b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:19.814085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129732c4-9b23-4a2b-a731-91f9e5814b38 None admin] PATH_INFO: `/v3/users/6f69eaa447474f11a3193c5a5b8dcd25` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:19.815897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129732c4-9b23-4a2b-a731-91f9e5814b38 None admin] RBAC: Authorizing `identity:get_user(user_id=6f69eaa447474f11a3193c5a5b8dcd25)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:19.817341 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129732c4-9b23-4a2b-a731-91f9e5814b38 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:19.819017 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 574/1151] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:02:19 2022] GET /identity/v3/users/6f69eaa447474f11a3193c5a5b8dcd25 => generated 287 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:19.824953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f2a29-27d2-4b5c-a99c-82340d705a47 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:19.928576 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f2a29-27d2-4b5c-a99c-82340d705a47 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:19.962230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932be23-e203-466c-8240-ef4643ce3491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:19.962899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932be23-e203-466c-8240-ef4643ce3491 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:19.963186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932be23-e203-466c-8240-ef4643ce3491 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:19.963492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932be23-e203-466c-8240-ef4643ce3491 None admin] PATH_INFO: `/v3/projects/92df8a206218448c9b0ab8f40e3196b4` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:19.964650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932be23-e203-466c-8240-ef4643ce3491 None admin] RBAC: Authorizing `identity:get_project(project_id=92df8a206218448c9b0ab8f40e3196b4)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:19.966095 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932be23-e203-466c-8240-ef4643ce3491 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:19.967817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 578/1152] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:02:19 2022] GET /identity/v3/projects/92df8a206218448c9b0ab8f40e3196b4 => generated 312 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:19.974456 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129732c4-9b23-4a2b-a731-91f9e5814b38 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:20.077154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129732c4-9b23-4a2b-a731-91f9e5814b38 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:20.112174 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19df51-5688-4052-9c05-d2e736d2e24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:20.112847 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19df51-5688-4052-9c05-d2e736d2e24f None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:20.113131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19df51-5688-4052-9c05-d2e736d2e24f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:20.113405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19df51-5688-4052-9c05-d2e736d2e24f None admin] PATH_INFO: `/v3/projects/92df8a206218448c9b0ab8f40e3196b4/users/6f69eaa447474f11a3193c5a5b8dcd25/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:20.115392 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19df51-5688-4052-9c05-d2e736d2e24f None admin] RBAC: Authorizing `identity:create_grant(project_id=92df8a206218448c9b0ab8f40e3196b4, user_id=6f69eaa447474f11a3193c5a5b8dcd25, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:20.116837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19df51-5688-4052-9c05-d2e736d2e24f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:20.131005 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 575/1153] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:02:19 2022] PUT /identity/v3/projects/92df8a206218448c9b0ab8f40e3196b4/users/6f69eaa447474f11a3193c5a5b8dcd25/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:02:22.157119 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18dc49-4a8e-4c9a-9dab-6467a6e44f6c None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:22.157695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18dc49-4a8e-4c9a-9dab-6467a6e44f6c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:22.157695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18dc49-4a8e-4c9a-9dab-6467a6e44f6c None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:22.159082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 579/1154] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:22.164308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd3a38-655f-4afa-a758-566b67c8c66e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:22.164794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd3a38-655f-4afa-a758-566b67c8c66e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:22.164891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd3a38-655f-4afa-a758-566b67c8c66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:22.183340 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-68fd3a38-655f-4afa-a758-566b67c8c66e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:22.184566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-68fd3a38-655f-4afa-a758-566b67c8c66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:22.202320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:22.256366 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 576/1155] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:22 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:22.279916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18dc49-4a8e-4c9a-9dab-6467a6e44f6c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:22.299283 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:22.383954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18dc49-4a8e-4c9a-9dab-6467a6e44f6c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:22.417841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425c020-90d0-4bfa-8383-8af6d7633477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:22.418388 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425c020-90d0-4bfa-8383-8af6d7633477 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:22.418700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425c020-90d0-4bfa-8383-8af6d7633477 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:22.418961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425c020-90d0-4bfa-8383-8af6d7633477 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:22.429580 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d425c020-90d0-4bfa-8383-8af6d7633477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-d425c020-90d0-4bfa-8383-8af6d7633477 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:02:22.434484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:02:22.435767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:02:22.437001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:02:22.437001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:02:22.437001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:02:22.437001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:02:22.437001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:02:22.437001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:02:22.437001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:02:22.437001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:02:22.437001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:02:22.437001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:02:22.437001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:02:22.437001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:02:22.437001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 580/1156] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:02:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:02:22.442330 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd3a38-655f-4afa-a758-566b67c8c66e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:22.544315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd3a38-655f-4afa-a758-566b67c8c66e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:22.579984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25b38f-f6d3-4870-9ad7-717c25c05d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:22.580725 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b38f-f6d3-4870-9ad7-717c25c05d64 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:22.581007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b38f-f6d3-4870-9ad7-717c25c05d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:22.581281 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25b38f-f6d3-4870-9ad7-717c25c05d64 None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:22.581996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25b38f-f6d3-4870-9ad7-717c25c05d64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:22.582302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25b38f-f6d3-4870-9ad7-717c25c05d64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:22.583871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25b38f-f6d3-4870-9ad7-717c25c05d64 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:22.592631 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 577/1157] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:02:22 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:22.598619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425c020-90d0-4bfa-8383-8af6d7633477 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:22.680188 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:22.705866 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425c020-90d0-4bfa-8383-8af6d7633477 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:22.740924 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499eb19-cc74-4362-91f8-71c580ab087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:22.741568 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499eb19-cc74-4362-91f8-71c580ab087e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:22.741847 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499eb19-cc74-4362-91f8-71c580ab087e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:22.742117 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499eb19-cc74-4362-91f8-71c580ab087e None admin] PATH_INFO: `/v3/users/6f69eaa447474f11a3193c5a5b8dcd25` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:22.743302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499eb19-cc74-4362-91f8-71c580ab087e None admin] RBAC: Authorizing `identity:get_user(user_id=6f69eaa447474f11a3193c5a5b8dcd25)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:22.744736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499eb19-cc74-4362-91f8-71c580ab087e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:22.746350 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 581/1158] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:02:22 2022] GET /identity/v3/users/6f69eaa447474f11a3193c5a5b8dcd25 => generated 287 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:22.755339 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25b38f-f6d3-4870-9ad7-717c25c05d64 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:22.772056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:22.858221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25b38f-f6d3-4870-9ad7-717c25c05d64 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:22.892217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc0afb-fc63-446a-9185-b9a5fd7bb2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:22.892876 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc0afb-fc63-446a-9185-b9a5fd7bb2fc None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:22.893162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc0afb-fc63-446a-9185-b9a5fd7bb2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:22.893440 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc0afb-fc63-446a-9185-b9a5fd7bb2fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:22.963787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc0afb-fc63-446a-9185-b9a5fd7bb2fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:22.965502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc0afb-fc63-446a-9185-b9a5fd7bb2fc None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:22.982988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:23.068277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 578/1159] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:02:22 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:02:23.074203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499eb19-cc74-4362-91f8-71c580ab087e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:23.180251 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499eb19-cc74-4362-91f8-71c580ab087e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:23.221615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9b74f-6c15-4037-9c72-15028f2abe97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:23.222171 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9b74f-6c15-4037-9c72-15028f2abe97 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:23.222451 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9b74f-6c15-4037-9c72-15028f2abe97 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:23.222755 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9b74f-6c15-4037-9c72-15028f2abe97 None admin] PATH_INFO: `/v3/projects/92df8a206218448c9b0ab8f40e3196b4` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:23.223915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9b74f-6c15-4037-9c72-15028f2abe97 None admin] RBAC: Authorizing `identity:get_project(project_id=92df8a206218448c9b0ab8f40e3196b4)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:23.225354 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9b74f-6c15-4037-9c72-15028f2abe97 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:23.227056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 582/1160] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:02:23 2022] GET /identity/v3/projects/92df8a206218448c9b0ab8f40e3196b4 => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:23.233176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc0afb-fc63-446a-9185-b9a5fd7bb2fc None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:23.343322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc0afb-fc63-446a-9185-b9a5fd7bb2fc None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:23.358540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:23.379068 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173aafc-6131-489d-aa6b-5958eb87a4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:23.379690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173aafc-6131-489d-aa6b-5958eb87a4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:23.379975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173aafc-6131-489d-aa6b-5958eb87a4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:23.380250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173aafc-6131-489d-aa6b-5958eb87a4c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:23.381041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173aafc-6131-489d-aa6b-5958eb87a4c8 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=92df8a206218448c9b0ab8f40e3196b4, user.id=6f69eaa447474f11a3193c5a5b8dcd25) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:23.381352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173aafc-6131-489d-aa6b-5958eb87a4c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:23.382815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173aafc-6131-489d-aa6b-5958eb87a4c8 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:23.388872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 579/1161] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:02:23 2022] GET /identity/v3/role_assignments?user.id=6f69eaa447474f11a3193c5a5b8dcd25&scope.project.id=92df8a206218448c9b0ab8f40e3196b4&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 610 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:25.475915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805deb01-1c52-47a0-b0ba-c7d6600c19b3 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:25.476458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805deb01-1c52-47a0-b0ba-c7d6600c19b3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:25.476458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805deb01-1c52-47a0-b0ba-c7d6600c19b3 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:25.477834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 583/1162] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:25.483380 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb7df2-6dcc-40cb-a030-6cb9cd6a69c4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:25.483886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb7df2-6dcc-40cb-a030-6cb9cd6a69c4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:25.484027 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb7df2-6dcc-40cb-a030-6cb9cd6a69c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:25.501941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9ccb7df2-6dcc-40cb-a030-6cb9cd6a69c4 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:25.503188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9ccb7df2-6dcc-40cb-a030-6cb9cd6a69c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:25.573783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 580/1163] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:25 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:25.597505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805deb01-1c52-47a0-b0ba-c7d6600c19b3 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:25.613223 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:25.700802 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805deb01-1c52-47a0-b0ba-c7d6600c19b3 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:25.737530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b445de-2fc5-47df-ac79-b442222d1aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:25.738061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b445de-2fc5-47df-ac79-b442222d1aee None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:25.738345 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b445de-2fc5-47df-ac79-b442222d1aee None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:25.738619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b445de-2fc5-47df-ac79-b442222d1aee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:25.806501 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b445de-2fc5-47df-ac79-b442222d1aee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:25.807994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b445de-2fc5-47df-ac79-b442222d1aee None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:25.822145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:25.910241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 584/1164] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:02:25 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:02:25.916562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb7df2-6dcc-40cb-a030-6cb9cd6a69c4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:26.019995 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb7df2-6dcc-40cb-a030-6cb9cd6a69c4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:26.029293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:26.058985 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7229a-3bf8-42f5-ba75-fc4f3dd7e5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:26.059638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7229a-3bf8-42f5-ba75-fc4f3dd7e5cd None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:26.059920 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7229a-3bf8-42f5-ba75-fc4f3dd7e5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:26.060194 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7229a-3bf8-42f5-ba75-fc4f3dd7e5cd None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:26.070042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7229a-3bf8-42f5-ba75-fc4f3dd7e5cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:26.071542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7229a-3bf8-42f5-ba75-fc4f3dd7e5cd None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-b5f7229a-3bf8-42f5-ba75-fc4f3dd7e5cd None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:02:26.083956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:02:26.085446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:02:26.086907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 581/1165] 10.222.0.100 () {56 vars in 1122 bytes} [Mon Aug 1 15:02:25 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:02:26.091566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b445de-2fc5-47df-ac79-b442222d1aee None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:26.194775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b445de-2fc5-47df-ac79-b442222d1aee None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:26.208983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:26.231627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79869b49-4986-4af3-ad6f-6acc09e8d345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:26.232245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869b49-4986-4af3-ad6f-6acc09e8d345 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:26.232528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869b49-4986-4af3-ad6f-6acc09e8d345 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:26.232802 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79869b49-4986-4af3-ad6f-6acc09e8d345 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:26.233501 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79869b49-4986-4af3-ad6f-6acc09e8d345 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:26.233815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79869b49-4986-4af3-ad6f-6acc09e8d345 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:26.235238 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79869b49-4986-4af3-ad6f-6acc09e8d345 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:26.243075 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 585/1166] 10.222.0.100 () {56 vars in 1120 bytes} [Mon Aug 1 15:02:26 2022] GET /identity/v3/domains?name=swift_test => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:28.243503 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3ec48-76f5-4418-b99e-cad88f140151 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:28.244071 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3ec48-76f5-4418-b99e-cad88f140151 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:28.244071 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb3ec48-76f5-4418-b99e-cad88f140151 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:28.245422 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 582/1167] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:28.250993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40ae75-6f15-48c8-badb-c484d961b400 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:28.251449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40ae75-6f15-48c8-badb-c484d961b400 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:28.251574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40ae75-6f15-48c8-badb-c484d961b400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:28.270102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-8a40ae75-6f15-48c8-badb-c484d961b400 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:28.271339 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8a40ae75-6f15-48c8-badb-c484d961b400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:28.340615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 586/1168] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:28 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:28.363883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3ec48-76f5-4418-b99e-cad88f140151 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:28.466747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb3ec48-76f5-4418-b99e-cad88f140151 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:28.500479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cc290-daaf-4564-a294-9b9d704fbeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:28.501016 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cc290-daaf-4564-a294-9b9d704fbeec None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:28.501296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cc290-daaf-4564-a294-9b9d704fbeec None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:28.501570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cc290-daaf-4564-a294-9b9d704fbeec None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:28.502863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cc290-daaf-4564-a294-9b9d704fbeec None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:28.504431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cc290-daaf-4564-a294-9b9d704fbeec None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:28.525989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 583/1169] 10.222.0.100 () {60 vars in 1143 bytes} [Mon Aug 1 15:02:28 2022] POST /identity/v3/domains => generated 268 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:02:30.579178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909951aa-c96c-4478-9eb3-d56cc49fca93 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:30.580210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909951aa-c96c-4478-9eb3-d56cc49fca93 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:30.580305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909951aa-c96c-4478-9eb3-d56cc49fca93 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:30.581634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 587/1170] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:30 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:30.587393 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fb6d5-3510-4d16-935d-40368a39eb95 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:30.587857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fb6d5-3510-4d16-935d-40368a39eb95 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:30.587948 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fb6d5-3510-4d16-935d-40368a39eb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:30.606839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e66fb6d5-3510-4d16-935d-40368a39eb95 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:30.608047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e66fb6d5-3510-4d16-935d-40368a39eb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:30.632134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:30.678182 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 584/1171] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:30 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:30.701385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909951aa-c96c-4478-9eb3-d56cc49fca93 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:30.804021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909951aa-c96c-4478-9eb3-d56cc49fca93 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:30.811512 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:30.841532 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7e2d1-daef-44fa-8ed7-1d440e82d42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:30.842176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e2d1-daef-44fa-8ed7-1d440e82d42e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:30.842458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e2d1-daef-44fa-8ed7-1d440e82d42e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:30.842764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e2d1-daef-44fa-8ed7-1d440e82d42e None admin] PATH_INFO: `/v3/domains/a3aef9f31ad64db8a1023048958848b3` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:30.853485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7e2d1-daef-44fa-8ed7-1d440e82d42e None admin] RBAC: Authorizing `identity:get_domain(domain_id=a3aef9f31ad64db8a1023048958848b3)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:30.854977 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7e2d1-daef-44fa-8ed7-1d440e82d42e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:30.856665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 588/1172] 10.222.0.100 () {56 vars in 1188 bytes} [Mon Aug 1 15:02:30 2022] GET /identity/v3/domains/a3aef9f31ad64db8a1023048958848b3 => generated 268 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:30.863566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fb6d5-3510-4d16-935d-40368a39eb95 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:30.967060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fb6d5-3510-4d16-935d-40368a39eb95 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:31.007419 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146bd1f-adac-4454-a47b-c747d354b1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:31.008041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146bd1f-adac-4454-a47b-c747d354b1a4 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:31.008325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146bd1f-adac-4454-a47b-c747d354b1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:31.008607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146bd1f-adac-4454-a47b-c747d354b1a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:31.009313 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146bd1f-adac-4454-a47b-c747d354b1a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:31.010791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146bd1f-adac-4454-a47b-c747d354b1a4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:31.037471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 585/1173] 10.222.0.100 () {60 vars in 1146 bytes} [Mon Aug 1 15:02:30 2022] POST /identity/v3/projects => generated 362 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:02:33.094738 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53b7d6-bf86-45c5-aa4d-6011e7022475 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:33.095331 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53b7d6-bf86-45c5-aa4d-6011e7022475 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:33.095331 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53b7d6-bf86-45c5-aa4d-6011e7022475 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:33.096665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 589/1174] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:33.102776 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3964388-af60-497b-bd96-913aaeb67a2e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:33.103226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3964388-af60-497b-bd96-913aaeb67a2e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:33.103318 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3964388-af60-497b-bd96-913aaeb67a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:33.130716 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e3964388-af60-497b-bd96-913aaeb67a2e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:33.131934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e3964388-af60-497b-bd96-913aaeb67a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:33.134866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:33.202105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 586/1175] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:33 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:33.227826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53b7d6-bf86-45c5-aa4d-6011e7022475 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:33.333007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53b7d6-bf86-45c5-aa4d-6011e7022475 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:33.367524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b5a88-ba62-49fa-87eb-1bebfb5288a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:33.368082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b5a88-ba62-49fa-87eb-1bebfb5288a9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:33.368366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b5a88-ba62-49fa-87eb-1bebfb5288a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:33.368642 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b5a88-ba62-49fa-87eb-1bebfb5288a9 None admin] PATH_INFO: `/v3/domains/a3aef9f31ad64db8a1023048958848b3` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:33.369875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b5a88-ba62-49fa-87eb-1bebfb5288a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a3aef9f31ad64db8a1023048958848b3)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:33.371359 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0b5a88-ba62-49fa-87eb-1bebfb5288a9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:33.372996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 590/1176] 10.222.0.100 () {56 vars in 1188 bytes} [Mon Aug 1 15:02:33 2022] GET /identity/v3/domains/a3aef9f31ad64db8a1023048958848b3 => generated 268 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:33.379504 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3964388-af60-497b-bd96-913aaeb67a2e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:33.479176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3964388-af60-497b-bd96-913aaeb67a2e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:33.513606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15681c92-8d0d-43c4-9947-697a8b4b460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:33.514233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15681c92-8d0d-43c4-9947-697a8b4b460b None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:33.514517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15681c92-8d0d-43c4-9947-697a8b4b460b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:33.514826 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15681c92-8d0d-43c4-9947-697a8b4b460b None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:33.515531 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15681c92-8d0d-43c4-9947-697a8b4b460b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:33.517001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15681c92-8d0d-43c4-9947-697a8b4b460b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:33.529283 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:33.541798 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 587/1177] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:02:33 2022] POST /identity/v3/users => generated 312 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:02:35.606189 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3750614-6820-4d64-ac28-ea1a8923fcb8 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:35.606757 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3750614-6820-4d64-ac28-ea1a8923fcb8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:35.606876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3750614-6820-4d64-ac28-ea1a8923fcb8 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:35.608206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 591/1178] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:35.614430 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8008dd3-9871-4938-9a3b-2cc02d9b950a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:35.614902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8008dd3-9871-4938-9a3b-2cc02d9b950a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:35.615007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8008dd3-9871-4938-9a3b-2cc02d9b950a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:35.633843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c8008dd3-9871-4938-9a3b-2cc02d9b950a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:35.635084 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c8008dd3-9871-4938-9a3b-2cc02d9b950a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:35.707632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 588/1179] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:35 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:35.735103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3750614-6820-4d64-ac28-ea1a8923fcb8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:35.771998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:35.843990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3750614-6820-4d64-ac28-ea1a8923fcb8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:35.877973 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb0910-0808-442c-807e-344e25a6cef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:35.878699 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb0910-0808-442c-807e-344e25a6cef0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:35.878983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb0910-0808-442c-807e-344e25a6cef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:35.879254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb0910-0808-442c-807e-344e25a6cef0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:35.899402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb0910-0808-442c-807e-344e25a6cef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-1efb0910-0808-442c-807e-344e25a6cef0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:02:35.904323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:02:35.905459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:02:35.906623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:02:35.906623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:02:35.906623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:02:35.906623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:02:35.906623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:02:35.906623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:02:35.906623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:02:35.906623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:02:35.906623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:02:35.906623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:02:35.906623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:02:35.906623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:02:35.906623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 592/1180] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:02:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:02:35.911873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8008dd3-9871-4938-9a3b-2cc02d9b950a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:36.010736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8008dd3-9871-4938-9a3b-2cc02d9b950a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:36.040530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:36.050647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db11aa-8270-4a1d-bd7d-3b46a8118a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:36.051302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db11aa-8270-4a1d-bd7d-3b46a8118a2b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:36.051588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db11aa-8270-4a1d-bd7d-3b46a8118a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:36.051866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db11aa-8270-4a1d-bd7d-3b46a8118a2b None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:36.052598 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db11aa-8270-4a1d-bd7d-3b46a8118a2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:36.052909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db11aa-8270-4a1d-bd7d-3b46a8118a2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:36.054299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db11aa-8270-4a1d-bd7d-3b46a8118a2b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:36.062960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 589/1181] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:02:35 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:36.073748 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb0910-0808-442c-807e-344e25a6cef0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:36.175175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:36.182065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb0910-0808-442c-807e-344e25a6cef0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:36.215753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e1fa0-c7a0-460b-af86-7df9f1b25f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:36.216402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e1fa0-c7a0-460b-af86-7df9f1b25f29 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:36.216687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e1fa0-c7a0-460b-af86-7df9f1b25f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:36.216967 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e1fa0-c7a0-460b-af86-7df9f1b25f29 None admin] PATH_INFO: `/v3/users/685cc503bbec4c64aa71bb35fc05c9d9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:36.231158 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e1fa0-c7a0-460b-af86-7df9f1b25f29 None admin] RBAC: Authorizing `identity:get_user(user_id=685cc503bbec4c64aa71bb35fc05c9d9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:36.232625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e1fa0-c7a0-460b-af86-7df9f1b25f29 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:36.234341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 593/1182] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:02:36 2022] GET /identity/v3/users/685cc503bbec4c64aa71bb35fc05c9d9 => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:36.241103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db11aa-8270-4a1d-bd7d-3b46a8118a2b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:36.346811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db11aa-8270-4a1d-bd7d-3b46a8118a2b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:36.387199 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365dbee0-98da-4b2b-ac9b-acc86869e652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:36.387812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365dbee0-98da-4b2b-ac9b-acc86869e652 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:36.388090 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365dbee0-98da-4b2b-ac9b-acc86869e652 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:36.388360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365dbee0-98da-4b2b-ac9b-acc86869e652 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:36.445740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:36.459416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365dbee0-98da-4b2b-ac9b-acc86869e652 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:36.460875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365dbee0-98da-4b2b-ac9b-acc86869e652 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:36.561236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 590/1183] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:02:36 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:02:36.567989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e1fa0-c7a0-460b-af86-7df9f1b25f29 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:36.671105 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e1fa0-c7a0-460b-af86-7df9f1b25f29 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:36.706648 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb71cd-7db0-4170-a933-27be9a4bdc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:36.707267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb71cd-7db0-4170-a933-27be9a4bdc35 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:36.707544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb71cd-7db0-4170-a933-27be9a4bdc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:36.707817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb71cd-7db0-4170-a933-27be9a4bdc35 None admin] PATH_INFO: `/v3/projects/ea86de4cef654b2bb0ce0af72a8b8120` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:36.708971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb71cd-7db0-4170-a933-27be9a4bdc35 None admin] RBAC: Authorizing `identity:get_project(project_id=ea86de4cef654b2bb0ce0af72a8b8120)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:36.710389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb71cd-7db0-4170-a933-27be9a4bdc35 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:36.712348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 594/1184] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:02:36 2022] GET /identity/v3/projects/ea86de4cef654b2bb0ce0af72a8b8120 => generated 362 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:36.718478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365dbee0-98da-4b2b-ac9b-acc86869e652 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:36.815553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:36.830946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365dbee0-98da-4b2b-ac9b-acc86869e652 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:36.864793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65856eed-bfae-4558-8a95-5a5786afaf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:36.865417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65856eed-bfae-4558-8a95-5a5786afaf3f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:36.865702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65856eed-bfae-4558-8a95-5a5786afaf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:36.865977 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65856eed-bfae-4558-8a95-5a5786afaf3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:36.866811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65856eed-bfae-4558-8a95-5a5786afaf3f None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=ea86de4cef654b2bb0ce0af72a8b8120, user.id=685cc503bbec4c64aa71bb35fc05c9d9) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:36.867128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65856eed-bfae-4558-8a95-5a5786afaf3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:36.868566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65856eed-bfae-4558-8a95-5a5786afaf3f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:36.874621 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 591/1185] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:02:36 2022] GET /identity/v3/role_assignments?user.id=685cc503bbec4c64aa71bb35fc05c9d9&scope.project.id=ea86de4cef654b2bb0ce0af72a8b8120&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:38.878900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c709d5-5b53-4a57-8cfd-49f884e7eca1 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:38.879446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c709d5-5b53-4a57-8cfd-49f884e7eca1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:38.879446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c709d5-5b53-4a57-8cfd-49f884e7eca1 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:38.880802 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 595/1186] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:38.887057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4f658-5df7-41c8-bb5b-5678f07fdf4e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:38.887558 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4f658-5df7-41c8-bb5b-5678f07fdf4e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:38.887651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4f658-5df7-41c8-bb5b-5678f07fdf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:38.905653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2cd4f658-5df7-41c8-bb5b-5678f07fdf4e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:38.906890 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2cd4f658-5df7-41c8-bb5b-5678f07fdf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:38.976872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 592/1187] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:38 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:38.999503 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c709d5-5b53-4a57-8cfd-49f884e7eca1 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:39.021983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:39.114681 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c709d5-5b53-4a57-8cfd-49f884e7eca1 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:39.148806 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7e4d-36aa-4377-aca6-4ee283df2c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:39.149362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7e4d-36aa-4377-aca6-4ee283df2c76 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:39.149646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7e4d-36aa-4377-aca6-4ee283df2c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:39.149927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7e4d-36aa-4377-aca6-4ee283df2c76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:39.160940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a7e4d-36aa-4377-aca6-4ee283df2c76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-5d7a7e4d-36aa-4377-aca6-4ee283df2c76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:02:39.165910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:02:39.167092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:02:39.167959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:02:39.167959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:02:39.167959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:02:39.167959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:02:39.167959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:02:39.167959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:02:39.167959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:02:39.167959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:02:39.167959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:02:39.167959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:02:39.167959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:02:39.167959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:02:39.167959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 596/1188] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:02:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:02:39.173996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4f658-5df7-41c8-bb5b-5678f07fdf4e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:39.257076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:39.275840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4f658-5df7-41c8-bb5b-5678f07fdf4e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:39.313397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27d874-d6fb-46bc-ade5-8a0e0112c0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:39.314048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27d874-d6fb-46bc-ade5-8a0e0112c0cb None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:39.314332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27d874-d6fb-46bc-ade5-8a0e0112c0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:39.314607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27d874-d6fb-46bc-ade5-8a0e0112c0cb None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:39.315350 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27d874-d6fb-46bc-ade5-8a0e0112c0cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:39.315658 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27d874-d6fb-46bc-ade5-8a0e0112c0cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:39.317035 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27d874-d6fb-46bc-ade5-8a0e0112c0cb None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:39.325424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 593/1189] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:02:39 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:39.333366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7e4d-36aa-4377-aca6-4ee283df2c76 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:39.418713 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:39.444169 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7e4d-36aa-4377-aca6-4ee283df2c76 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:39.479763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8200009-0012-4ad2-97ca-01667afb90f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:39.480411 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8200009-0012-4ad2-97ca-01667afb90f9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:39.480694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8200009-0012-4ad2-97ca-01667afb90f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:39.480972 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8200009-0012-4ad2-97ca-01667afb90f9 None admin] PATH_INFO: `/v3/users/685cc503bbec4c64aa71bb35fc05c9d9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:39.482215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8200009-0012-4ad2-97ca-01667afb90f9 None admin] RBAC: Authorizing `identity:get_user(user_id=685cc503bbec4c64aa71bb35fc05c9d9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:39.483684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8200009-0012-4ad2-97ca-01667afb90f9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:39.485302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 597/1190] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:02:39 2022] GET /identity/v3/users/685cc503bbec4c64aa71bb35fc05c9d9 => generated 312 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:39.491151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27d874-d6fb-46bc-ade5-8a0e0112c0cb None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:39.590903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27d874-d6fb-46bc-ade5-8a0e0112c0cb None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:39.632646 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e9abb-1f90-41db-a9a4-a54642a7602d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:39.633312 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e9abb-1f90-41db-a9a4-a54642a7602d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:39.633602 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e9abb-1f90-41db-a9a4-a54642a7602d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:39.633875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e9abb-1f90-41db-a9a4-a54642a7602d None admin] PATH_INFO: `/v3/projects/ea86de4cef654b2bb0ce0af72a8b8120` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:39.635116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e9abb-1f90-41db-a9a4-a54642a7602d None admin] RBAC: Authorizing `identity:get_project(project_id=ea86de4cef654b2bb0ce0af72a8b8120)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:39.636572 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319e9abb-1f90-41db-a9a4-a54642a7602d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:39.638256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 594/1191] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:02:39 2022] GET /identity/v3/projects/ea86de4cef654b2bb0ce0af72a8b8120 => generated 362 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:39.644881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8200009-0012-4ad2-97ca-01667afb90f9 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:39.748583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8200009-0012-4ad2-97ca-01667afb90f9 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:39.783317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c7dc4-a1b3-4491-94f5-5a15ced67e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:39.783979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c7dc4-a1b3-4491-94f5-5a15ced67e25 None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:39.784252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c7dc4-a1b3-4491-94f5-5a15ced67e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:39.784523 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c7dc4-a1b3-4491-94f5-5a15ced67e25 None admin] PATH_INFO: `/v3/projects/ea86de4cef654b2bb0ce0af72a8b8120/users/685cc503bbec4c64aa71bb35fc05c9d9/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:39.786442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c7dc4-a1b3-4491-94f5-5a15ced67e25 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea86de4cef654b2bb0ce0af72a8b8120, user_id=685cc503bbec4c64aa71bb35fc05c9d9, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:39.787921 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c7dc4-a1b3-4491-94f5-5a15ced67e25 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:39.801681 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 598/1192] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:02:39 2022] PUT /identity/v3/projects/ea86de4cef654b2bb0ce0af72a8b8120/users/685cc503bbec4c64aa71bb35fc05c9d9/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:02:41.794354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890bc1b-5ae6-47f2-bfa2-b0e296741d1d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:41.794982 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890bc1b-5ae6-47f2-bfa2-b0e296741d1d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:41.794982 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890bc1b-5ae6-47f2-bfa2-b0e296741d1d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:41.796290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 595/1193] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:02:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:02:41.801633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc1615-59b6-4be3-8c53-400964f3d40a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:41.802078 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc1615-59b6-4be3-8c53-400964f3d40a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:41.802193 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc1615-59b6-4be3-8c53-400964f3d40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:41.820302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-37fc1615-59b6-4be3-8c53-400964f3d40a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:02:41.821538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-37fc1615-59b6-4be3-8c53-400964f3d40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:02:41.823774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:41.890835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 599/1194] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:02:41 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:02:41.915542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890bc1b-5ae6-47f2-bfa2-b0e296741d1d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:41.931337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:42.022877 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890bc1b-5ae6-47f2-bfa2-b0e296741d1d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:42.058409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e259ff0-2d54-4a63-8f1d-0ae96b502e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:42.059012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e259ff0-2d54-4a63-8f1d-0ae96b502e1f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:42.059298 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e259ff0-2d54-4a63-8f1d-0ae96b502e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:42.059573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e259ff0-2d54-4a63-8f1d-0ae96b502e1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:42.070483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e259ff0-2d54-4a63-8f1d-0ae96b502e1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-0e259ff0-2d54-4a63-8f1d-0ae96b502e1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:02:42.075442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:02:42.076386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:02:42.077252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:02:42.077252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:02:42.077252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:02:42.077252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:02:42.077252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:02:42.077252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:02:42.077252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:02:42.077252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:02:42.077252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:02:42.077252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:02:42.077252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:02:42.077252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:02:42.077252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 596/1195] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:02:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:02:42.082816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc1615-59b6-4be3-8c53-400964f3d40a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:42.182564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc1615-59b6-4be3-8c53-400964f3d40a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:42.216213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d05db-33ba-4da6-bfd5-1db40704c45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:42.216843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d05db-33ba-4da6-bfd5-1db40704c45e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:42.217124 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d05db-33ba-4da6-bfd5-1db40704c45e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:42.217407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d05db-33ba-4da6-bfd5-1db40704c45e None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:42.218118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d05db-33ba-4da6-bfd5-1db40704c45e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:42.218425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d05db-33ba-4da6-bfd5-1db40704c45e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:42.220348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d05db-33ba-4da6-bfd5-1db40704c45e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:42.226872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:42.230545 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 600/1196] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:02:42 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:42.242990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e259ff0-2d54-4a63-8f1d-0ae96b502e1f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:42.312545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:42.343562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e259ff0-2d54-4a63-8f1d-0ae96b502e1f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:42.377862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518f719-bfae-44de-8341-d54b20a7332a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:42.378507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518f719-bfae-44de-8341-d54b20a7332a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:42.378823 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518f719-bfae-44de-8341-d54b20a7332a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:42.379100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518f719-bfae-44de-8341-d54b20a7332a None admin] PATH_INFO: `/v3/users/685cc503bbec4c64aa71bb35fc05c9d9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:42.380282 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518f719-bfae-44de-8341-d54b20a7332a None admin] RBAC: Authorizing `identity:get_user(user_id=685cc503bbec4c64aa71bb35fc05c9d9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:42.381717 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518f719-bfae-44de-8341-d54b20a7332a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:42.383359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 597/1197] 10.222.0.100 () {56 vars in 1182 bytes} [Mon Aug 1 15:02:42 2022] GET /identity/v3/users/685cc503bbec4c64aa71bb35fc05c9d9 => generated 312 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:42.389178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d05db-33ba-4da6-bfd5-1db40704c45e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:42.496115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d05db-33ba-4da6-bfd5-1db40704c45e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:42.531823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6f1ec-f801-49cc-8adb-0556c431490c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:42.532466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6f1ec-f801-49cc-8adb-0556c431490c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:42.532752 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6f1ec-f801-49cc-8adb-0556c431490c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:42.533028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6f1ec-f801-49cc-8adb-0556c431490c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:42.605248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:42.607747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6f1ec-f801-49cc-8adb-0556c431490c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:42.609596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6f1ec-f801-49cc-8adb-0556c431490c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:42.709875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 601/1198] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:02:42 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:02:42.716051 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518f719-bfae-44de-8341-d54b20a7332a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:42.815177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518f719-bfae-44de-8341-d54b20a7332a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:42.851673 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67529b4-0325-4b7e-8653-c9ab2aa07e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:42.852316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67529b4-0325-4b7e-8653-c9ab2aa07e0b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:42.852596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67529b4-0325-4b7e-8653-c9ab2aa07e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:42.852870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67529b4-0325-4b7e-8653-c9ab2aa07e0b None admin] PATH_INFO: `/v3/projects/ea86de4cef654b2bb0ce0af72a8b8120` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:42.854025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67529b4-0325-4b7e-8653-c9ab2aa07e0b None admin] RBAC: Authorizing `identity:get_project(project_id=ea86de4cef654b2bb0ce0af72a8b8120)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:42.855544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67529b4-0325-4b7e-8653-c9ab2aa07e0b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:42.857151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 598/1199] 10.222.0.100 () {56 vars in 1191 bytes} [Mon Aug 1 15:02:42 2022] GET /identity/v3/projects/ea86de4cef654b2bb0ce0af72a8b8120 => generated 362 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:42.864509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6f1ec-f801-49cc-8adb-0556c431490c None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:02:42.964217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:42.967707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6f1ec-f801-49cc-8adb-0556c431490c None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:02:43.002924 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047a486-f16f-43e4-ae52-ce234f3b0ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:02:43.003543 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047a486-f16f-43e4-ae52-ce234f3b0ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:02:43.003829 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047a486-f16f-43e4-ae52-ce234f3b0ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:02:43.004110 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047a486-f16f-43e4-ae52-ce234f3b0ac9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:02:43.004896 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047a486-f16f-43e4-ae52-ce234f3b0ac9 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=ea86de4cef654b2bb0ce0af72a8b8120, user.id=685cc503bbec4c64aa71bb35fc05c9d9) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:02:43.005213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047a486-f16f-43e4-ae52-ce234f3b0ac9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:02:43.006649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047a486-f16f-43e4-ae52-ce234f3b0ac9 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:02:43.012454 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 602/1200] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:02:42 2022] GET /identity/v3/role_assignments?user.id=685cc503bbec4c64aa71bb35fc05c9d9&scope.project.id=ea86de4cef654b2bb0ce0af72a8b8120&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 610 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:02:52.847874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:02:53.009800 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=18 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:39.417292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442a33a-add8-493f-8082-0c13c9792136 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:39.417894 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442a33a-add8-493f-8082-0c13c9792136 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:39.417894 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f442a33a-add8-493f-8082-0c13c9792136 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:39.419614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 599/1201] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:03:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:03:39.425399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f866ba4-43ac-420b-af4b-742ec8a63208 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:39.425879 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f866ba4-43ac-420b-af4b-742ec8a63208 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:39.425977 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f866ba4-43ac-420b-af4b-742ec8a63208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:39.451594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5f866ba4-43ac-420b-af4b-742ec8a63208 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:39.458792 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5f866ba4-43ac-420b-af4b-742ec8a63208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:39.558167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 603/1202] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:03:39 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:03:39.583036 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442a33a-add8-493f-8082-0c13c9792136 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:39.718508 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f442a33a-add8-493f-8082-0c13c9792136 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:39.762147 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c607f-ce00-40fa-b33e-072da5a77b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:39.762843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c607f-ce00-40fa-b33e-072da5a77b69 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:39.763131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c607f-ce00-40fa-b33e-072da5a77b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:39.763410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c607f-ce00-40fa-b33e-072da5a77b69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:39.777517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c607f-ce00-40fa-b33e-072da5a77b69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:39.779144 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c607f-ce00-40fa-b33e-072da5a77b69 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-1b7c607f-ce00-40fa-b33e-072da5a77b69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:03:39.796413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:39.797533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:39.799941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:03:39.799941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:03:39.799941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:39.799941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:03:39.799941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 600/1203] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:03:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:39.804228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f866ba4-43ac-420b-af4b-742ec8a63208 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:39.952462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f866ba4-43ac-420b-af4b-742ec8a63208 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:39.991469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:40.000232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a16bce-d871-46e8-a6b7-530da15ad053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:40.000866 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a16bce-d871-46e8-a6b7-530da15ad053 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:40.001316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a16bce-d871-46e8-a6b7-530da15ad053 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:40.001602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a16bce-d871-46e8-a6b7-530da15ad053 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:40.002280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a16bce-d871-46e8-a6b7-530da15ad053 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:40.002591 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a16bce-d871-46e8-a6b7-530da15ad053 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:40.004007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a16bce-d871-46e8-a6b7-530da15ad053 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:40.020290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 604/1204] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:03:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:40.026040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c607f-ce00-40fa-b33e-072da5a77b69 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:40.104569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:40.175821 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c607f-ce00-40fa-b33e-072da5a77b69 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:40.215439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9cd47-2a53-4eb4-8053-52efee492970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:40.216091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9cd47-2a53-4eb4-8053-52efee492970 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:40.216376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9cd47-2a53-4eb4-8053-52efee492970 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:40.216661 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9cd47-2a53-4eb4-8053-52efee492970 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:40.217943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9cd47-2a53-4eb4-8053-52efee492970 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:40.219504 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9cd47-2a53-4eb4-8053-52efee492970 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:40.248962 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 601/1205] 10.222.0.100 () {60 vars in 1137 bytes} [Mon Aug 1 15:03:40 2022] POST /identity/v3/users => generated 252 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:03:42.804286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2733bc-5c3b-4c6f-b8e6-4743f7d548ab None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:42.804799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2733bc-5c3b-4c6f-b8e6-4743f7d548ab None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:42.804799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2733bc-5c3b-4c6f-b8e6-4743f7d548ab None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:42.806254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 605/1206] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:03:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:03:42.812652 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93649c75-3131-4244-b7b3-8a6d8276d3fb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:42.813114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93649c75-3131-4244-b7b3-8a6d8276d3fb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:42.813206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93649c75-3131-4244-b7b3-8a6d8276d3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:42.833597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-93649c75-3131-4244-b7b3-8a6d8276d3fb None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:42.834999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-93649c75-3131-4244-b7b3-8a6d8276d3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:42.916504 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 602/1207] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:03:42 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:03:42.940635 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2733bc-5c3b-4c6f-b8e6-4743f7d548ab None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:43.090338 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2733bc-5c3b-4c6f-b8e6-4743f7d548ab None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:43.136357 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e6e89-cae3-4b59-9845-a11b3a68e2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:43.137077 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e6e89-cae3-4b59-9845-a11b3a68e2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:43.137499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e6e89-cae3-4b59-9845-a11b3a68e2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:43.137789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7e6e89-cae3-4b59-9845-a11b3a68e2c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:43.151667 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7e6e89-cae3-4b59-9845-a11b3a68e2c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-9d7e6e89-cae3-4b59-9845-a11b3a68e2c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:03:43.157653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:43.158717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:43.159589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:43.159589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:43.159589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:43.159589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:43.159589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:03:43.159589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:03:43.159589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:03:43.159589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:03:43.159589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:03:43.159589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:03:43.159589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:03:43.159589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:03:43.159589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 606/1208] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:03:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:03:43.166925 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93649c75-3131-4244-b7b3-8a6d8276d3fb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:43.213854 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:43.283703 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93649c75-3131-4244-b7b3-8a6d8276d3fb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:43.328237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404c03f-9936-4d4a-bb0c-c96a21f2e9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:43.329324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404c03f-9936-4d4a-bb0c-c96a21f2e9be None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:43.329756 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404c03f-9936-4d4a-bb0c-c96a21f2e9be None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:43.330254 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404c03f-9936-4d4a-bb0c-c96a21f2e9be None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:43.331519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404c03f-9936-4d4a-bb0c-c96a21f2e9be None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:43.332047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404c03f-9936-4d4a-bb0c-c96a21f2e9be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:43.333803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404c03f-9936-4d4a-bb0c-c96a21f2e9be None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:43.344553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 603/1209] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:03:43 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:43.350300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e6e89-cae3-4b59-9845-a11b3a68e2c0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:43.415598 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:43.499139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7e6e89-cae3-4b59-9845-a11b3a68e2c0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:43.549587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985ef7f-b3ac-4808-8170-9087ee865c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:43.549587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985ef7f-b3ac-4808-8170-9087ee865c22 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:43.550506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985ef7f-b3ac-4808-8170-9087ee865c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:43.550506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985ef7f-b3ac-4808-8170-9087ee865c22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:43.563663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b985ef7f-b3ac-4808-8170-9087ee865c22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:43.565596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b985ef7f-b3ac-4808-8170-9087ee865c22 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-b985ef7f-b3ac-4808-8170-9087ee865c22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:03:43.585660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:43.586830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:43.591643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:03:43.591643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:03:43.591643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:43.591643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:03:43.591643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 607/1210] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:43.593901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404c03f-9936-4d4a-bb0c-c96a21f2e9be None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:43.728071 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:43.732846 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404c03f-9936-4d4a-bb0c-c96a21f2e9be None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:43.783067 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d628ad1-a967-46ed-a89f-0fe158411587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:43.783749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d628ad1-a967-46ed-a89f-0fe158411587 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:43.784034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d628ad1-a967-46ed-a89f-0fe158411587 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:43.784309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d628ad1-a967-46ed-a89f-0fe158411587 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:43.785012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d628ad1-a967-46ed-a89f-0fe158411587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:43.785323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d628ad1-a967-46ed-a89f-0fe158411587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:43.786828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d628ad1-a967-46ed-a89f-0fe158411587 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:43.802450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 604/1211] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:03:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:43.816224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985ef7f-b3ac-4808-8170-9087ee865c22 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:43.936858 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985ef7f-b3ac-4808-8170-9087ee865c22 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:43.938450 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:43.977235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c420bc-cc79-424f-8636-4f37fa91bb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:43.978462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c420bc-cc79-424f-8636-4f37fa91bb1e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:43.978462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c420bc-cc79-424f-8636-4f37fa91bb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:43.978462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c420bc-cc79-424f-8636-4f37fa91bb1e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:43.987370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c420bc-cc79-424f-8636-4f37fa91bb1e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:43.988790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c420bc-cc79-424f-8636-4f37fa91bb1e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-b4c420bc-cc79-424f-8636-4f37fa91bb1e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:03:44.001925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:44.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:44.004133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:44.004133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:44.004133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:03:44.004133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:03:44.004133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:03:44.004133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:03:44.004133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:03:44.004133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:03:44.004133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:03:44.004133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:03:44.004133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 608/1212] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:03:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:44.009537 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d628ad1-a967-46ed-a89f-0fe158411587 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:44.153593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d628ad1-a967-46ed-a89f-0fe158411587 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:44.210530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5513bc-8804-42cc-abec-a6dbbc3dadaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:44.211415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5513bc-8804-42cc-abec-a6dbbc3dadaf None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:44.211692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5513bc-8804-42cc-abec-a6dbbc3dadaf None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:44.211975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5513bc-8804-42cc-abec-a6dbbc3dadaf None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:44.212780 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5513bc-8804-42cc-abec-a6dbbc3dadaf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:44.213217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5513bc-8804-42cc-abec-a6dbbc3dadaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:44.215117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5513bc-8804-42cc-abec-a6dbbc3dadaf None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:44.238620 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 605/1213] 10.222.0.100 () {56 vars in 1148 bytes} [Mon Aug 1 15:03:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:44.244762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c420bc-cc79-424f-8636-4f37fa91bb1e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:44.373201 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c420bc-cc79-424f-8636-4f37fa91bb1e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:44.419762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d725c2-2369-4ab8-b40a-bc7a255c4f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:44.420926 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d725c2-2369-4ab8-b40a-bc7a255c4f0b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:44.421203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d725c2-2369-4ab8-b40a-bc7a255c4f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:44.421478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d725c2-2369-4ab8-b40a-bc7a255c4f0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:44.458035 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:44.501903 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d725c2-2369-4ab8-b40a-bc7a255c4f0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:44.503638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d725c2-2369-4ab8-b40a-bc7a255c4f0b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:44.635611 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 609/1214] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:03:44 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:03:44.637941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5513bc-8804-42cc-abec-a6dbbc3dadaf None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:44.661796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:44.781073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5513bc-8804-42cc-abec-a6dbbc3dadaf None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:44.823656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9a6cb-cb10-42ad-b425-e50716d34d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:44.824506 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9a6cb-cb10-42ad-b425-e50716d34d4d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:44.824890 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9a6cb-cb10-42ad-b425-e50716d34d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:44.825165 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9a6cb-cb10-42ad-b425-e50716d34d4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:44.839466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9a6cb-cb10-42ad-b425-e50716d34d4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:44.841113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9a6cb-cb10-42ad-b425-e50716d34d4d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-34b9a6cb-cb10-42ad-b425-e50716d34d4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:03:44.866117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:44.867345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:44.873527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:03:44.873527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:03:44.873527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:44.873527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:03:44.873527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 606/1215] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:44.876935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d725c2-2369-4ab8-b40a-bc7a255c4f0b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:44.960385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:45.016963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d725c2-2369-4ab8-b40a-bc7a255c4f0b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:45.068906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8d3db-165a-45c4-a754-4aed532ca460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:45.069543 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8d3db-165a-45c4-a754-4aed532ca460 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:45.069772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8d3db-165a-45c4-a754-4aed532ca460 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:45.070116 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8d3db-165a-45c4-a754-4aed532ca460 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:45.070836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8d3db-165a-45c4-a754-4aed532ca460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:45.071144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8d3db-165a-45c4-a754-4aed532ca460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:45.072714 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8d3db-165a-45c4-a754-4aed532ca460 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:45.094703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 610/1216] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:03:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:45.098344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9a6cb-cb10-42ad-b425-e50716d34d4d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:45.216226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:45.267596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9a6cb-cb10-42ad-b425-e50716d34d4d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:45.310045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f16cbd-4ca3-48ef-ba19-e8ed3e331766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:45.311457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f16cbd-4ca3-48ef-ba19-e8ed3e331766 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:45.311843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f16cbd-4ca3-48ef-ba19-e8ed3e331766 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:45.312219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f16cbd-4ca3-48ef-ba19-e8ed3e331766 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:45.320249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f16cbd-4ca3-48ef-ba19-e8ed3e331766 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:45.322002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f16cbd-4ca3-48ef-ba19-e8ed3e331766 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-76f16cbd-4ca3-48ef-ba19-e8ed3e331766 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:03:45.335871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:45.337048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:45.338167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:03:45.338167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:03:45.338167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:03:45.338167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:03:45.338167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:03:45.338167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:03:45.338167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:03:45.338167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:03:45.342997 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 607/1217] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:03:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:45.345654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8d3db-165a-45c4-a754-4aed532ca460 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:45.470557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8d3db-165a-45c4-a754-4aed532ca460 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:45.494645 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:45.510497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd90d444-63d4-473f-b61b-f379a0b4c06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:45.511473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90d444-63d4-473f-b61b-f379a0b4c06a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:45.511778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90d444-63d4-473f-b61b-f379a0b4c06a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:45.512059 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90d444-63d4-473f-b61b-f379a0b4c06a None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:45.512863 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90d444-63d4-473f-b61b-f379a0b4c06a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:45.513169 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90d444-63d4-473f-b61b-f379a0b4c06a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:45.514813 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd90d444-63d4-473f-b61b-f379a0b4c06a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:45.529413 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 611/1218] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:03:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:45.536624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f16cbd-4ca3-48ef-ba19-e8ed3e331766 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:45.678721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f16cbd-4ca3-48ef-ba19-e8ed3e331766 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:45.712910 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:45.719060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237a34d-ea54-4f3a-9e1e-210c44449bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:45.719688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237a34d-ea54-4f3a-9e1e-210c44449bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:45.719973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237a34d-ea54-4f3a-9e1e-210c44449bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:45.720250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237a34d-ea54-4f3a-9e1e-210c44449bc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:45.721046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237a34d-ea54-4f3a-9e1e-210c44449bc4 None admin] RBAC: Adding query filter params (role.id=70e0c42f1bdd49178db398637f021bbc, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=3bac5863e7bc4c7e854bd60912adb561) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:45.721376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237a34d-ea54-4f3a-9e1e-210c44449bc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:45.723549 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237a34d-ea54-4f3a-9e1e-210c44449bc4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:45.734707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 608/1219] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:03:45 2022] GET /identity/v3/role_assignments?user.id=3bac5863e7bc4c7e854bd60912adb561&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=70e0c42f1bdd49178db398637f021bbc => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:03:47.796004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1991d-d9b2-470f-8cc0-4602c0363e35 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:47.796513 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1991d-d9b2-470f-8cc0-4602c0363e35 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:47.796513 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1991d-d9b2-470f-8cc0-4602c0363e35 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:47.797824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 612/1220] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:03:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:03:47.803596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecab09-b812-431a-92b2-28f20dadab34 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:47.804502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecab09-b812-431a-92b2-28f20dadab34 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:47.804502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecab09-b812-431a-92b2-28f20dadab34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:47.828595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-15ecab09-b812-431a-92b2-28f20dadab34 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:47.829822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-15ecab09-b812-431a-92b2-28f20dadab34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:47.919986 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 609/1221] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:03:47 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:03:47.942758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1991d-d9b2-470f-8cc0-4602c0363e35 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:48.069854 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1991d-d9b2-470f-8cc0-4602c0363e35 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:48.110456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb590fd-aebf-4db4-931c-a350530e85cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:48.111555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb590fd-aebf-4db4-931c-a350530e85cb None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:48.111900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb590fd-aebf-4db4-931c-a350530e85cb None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:48.112292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb590fd-aebf-4db4-931c-a350530e85cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:48.129826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb590fd-aebf-4db4-931c-a350530e85cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-ebb590fd-aebf-4db4-931c-a350530e85cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:03:48.135721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:48.137016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:48.138259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:48.138259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:48.138259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:48.138259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:48.138259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:03:48.138259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:03:48.138259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:03:48.138259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:03:48.138259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:03:48.138259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:03:48.138259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:03:48.138259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:03:48.138259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 613/1222] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:03:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:48.143975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecab09-b812-431a-92b2-28f20dadab34 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:48.270527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecab09-b812-431a-92b2-28f20dadab34 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:48.296605 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:48.309596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44febd5b-f7a4-4ee7-9043-59ab270b856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:48.310928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44febd5b-f7a4-4ee7-9043-59ab270b856a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:48.311238 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44febd5b-f7a4-4ee7-9043-59ab270b856a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:48.311522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44febd5b-f7a4-4ee7-9043-59ab270b856a None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:48.312260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44febd5b-f7a4-4ee7-9043-59ab270b856a None admin] RBAC: Adding query filter params (name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:48.312570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44febd5b-f7a4-4ee7-9043-59ab270b856a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:48.314088 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44febd5b-f7a4-4ee7-9043-59ab270b856a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:48.323551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 610/1223] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:03:48 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:48.330068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb590fd-aebf-4db4-931c-a350530e85cb None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:48.378550 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:48.442605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb590fd-aebf-4db4-931c-a350530e85cb None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:48.478640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa37a5-f5d4-46d8-843f-db0f2a3e7035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:48.479318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa37a5-f5d4-46d8-843f-db0f2a3e7035 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:48.479604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa37a5-f5d4-46d8-843f-db0f2a3e7035 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:48.479879 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa37a5-f5d4-46d8-843f-db0f2a3e7035 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:48.492107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa37a5-f5d4-46d8-843f-db0f2a3e7035 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:48.493584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa37a5-f5d4-46d8-843f-db0f2a3e7035 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-c5aa37a5-f5d4-46d8-843f-db0f2a3e7035 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:03:48.510784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:48.512817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:48.514781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:03:48.514781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:03:48.514781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:48.514781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:03:48.514781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 614/1224] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:48.517936 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44febd5b-f7a4-4ee7-9043-59ab270b856a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:48.632064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44febd5b-f7a4-4ee7-9043-59ab270b856a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:48.670184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c9303-6146-4d7b-a0f8-607808aac7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:48.670841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c9303-6146-4d7b-a0f8-607808aac7e1 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:48.671122 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c9303-6146-4d7b-a0f8-607808aac7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:48.671422 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c9303-6146-4d7b-a0f8-607808aac7e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:48.672234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c9303-6146-4d7b-a0f8-607808aac7e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:48.672475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c9303-6146-4d7b-a0f8-607808aac7e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:48.673913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c9303-6146-4d7b-a0f8-607808aac7e1 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:48.688551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 611/1225] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:03:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:03:48.694398 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa37a5-f5d4-46d8-843f-db0f2a3e7035 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:48.810755 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:48.819046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa37a5-f5d4-46d8-843f-db0f2a3e7035 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:48.861353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf376e-c4b1-4d47-9097-fa7330668f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:48.862112 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf376e-c4b1-4d47-9097-fa7330668f29 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:48.862491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf376e-c4b1-4d47-9097-fa7330668f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:48.862697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf376e-c4b1-4d47-9097-fa7330668f29 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:48.871203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf376e-c4b1-4d47-9097-fa7330668f29 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:48.872674 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf376e-c4b1-4d47-9097-fa7330668f29 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-66cf376e-c4b1-4d47-9097-fa7330668f29 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:03:48.886001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:48.888014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:48.890034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:48.890034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:48.890034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:03:48.890034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:03:48.890034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:03:48.890034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:03:48.890034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:03:48.890034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:03:48.890034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:03:48.890034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:03:48.890034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 615/1226] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:03:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:48.894031 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c9303-6146-4d7b-a0f8-607808aac7e1 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:48.955582 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:49.026057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c9303-6146-4d7b-a0f8-607808aac7e1 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:49.071611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00adf9b-0253-43a4-9ad8-75fbc882bbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:49.072305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00adf9b-0253-43a4-9ad8-75fbc882bbd0 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:49.072588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00adf9b-0253-43a4-9ad8-75fbc882bbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:49.072876 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00adf9b-0253-43a4-9ad8-75fbc882bbd0 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:49.073658 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00adf9b-0253-43a4-9ad8-75fbc882bbd0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:49.073966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00adf9b-0253-43a4-9ad8-75fbc882bbd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:49.075431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00adf9b-0253-43a4-9ad8-75fbc882bbd0 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:49.096012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 612/1227] 10.222.0.100 () {56 vars in 1148 bytes} [Mon Aug 1 15:03:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:03:49.103207 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf376e-c4b1-4d47-9097-fa7330668f29 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:49.223629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf376e-c4b1-4d47-9097-fa7330668f29 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:49.260843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b09213-122c-4e55-9994-c40baf3edb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:49.261486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09213-122c-4e55-9994-c40baf3edb7b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:49.261770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09213-122c-4e55-9994-c40baf3edb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:49.262048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09213-122c-4e55-9994-c40baf3edb7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:49.273828 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b09213-122c-4e55-9994-c40baf3edb7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:49.275340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b09213-122c-4e55-9994-c40baf3edb7b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-d1b09213-122c-4e55-9994-c40baf3edb7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:03:49.291694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:49.292867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:49.294145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:03:49.294145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:03:49.294145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:49.294145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:03:49.294145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 616/1228] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:49.299356 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00adf9b-0253-43a4-9ad8-75fbc882bbd0 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:49.424364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00adf9b-0253-43a4-9ad8-75fbc882bbd0 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:49.427189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:49.470413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa600b41-7330-4180-8d52-59c5f0ae248f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:49.471144 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa600b41-7330-4180-8d52-59c5f0ae248f None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:49.471469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa600b41-7330-4180-8d52-59c5f0ae248f None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:49.471845 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa600b41-7330-4180-8d52-59c5f0ae248f None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:49.472687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa600b41-7330-4180-8d52-59c5f0ae248f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:49.473079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa600b41-7330-4180-8d52-59c5f0ae248f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:49.474873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa600b41-7330-4180-8d52-59c5f0ae248f None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:49.489071 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 613/1229] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:03:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:49.495353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b09213-122c-4e55-9994-c40baf3edb7b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:49.505949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:49.611774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b09213-122c-4e55-9994-c40baf3edb7b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:49.650094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00002413-2098-424e-8711-602726e79b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:49.650779 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00002413-2098-424e-8711-602726e79b6a None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:49.651250 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00002413-2098-424e-8711-602726e79b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:49.651663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00002413-2098-424e-8711-602726e79b6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:49.661703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00002413-2098-424e-8711-602726e79b6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:49.663443 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00002413-2098-424e-8711-602726e79b6a None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-00002413-2098-424e-8711-602726e79b6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:03:49.677569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:49.678614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:49.680103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:03:49.680103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:03:49.680103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:03:49.680103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:03:49.680103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:03:49.680103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:03:49.680103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:03:49.680103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:03:49.685239 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 617/1230] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:03:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:49.686810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa600b41-7330-4180-8d52-59c5f0ae248f None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:49.812129 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa600b41-7330-4180-8d52-59c5f0ae248f None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:49.852068 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3afca2-2a95-470a-9a3f-36ebe22d7766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:49.852831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3afca2-2a95-470a-9a3f-36ebe22d7766 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:49.853120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3afca2-2a95-470a-9a3f-36ebe22d7766 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:49.853462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3afca2-2a95-470a-9a3f-36ebe22d7766 None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:49.854571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3afca2-2a95-470a-9a3f-36ebe22d7766 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:49.855046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3afca2-2a95-470a-9a3f-36ebe22d7766 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:49.856892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3afca2-2a95-470a-9a3f-36ebe22d7766 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:49.884879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 614/1231] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:03:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:03:49.886057 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00002413-2098-424e-8711-602726e79b6a None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:49.961755 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:50.002291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00002413-2098-424e-8711-602726e79b6a None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:50.038297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee140cc-1e7b-414d-8abe-5afc0cb4ea8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:50.039089 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee140cc-1e7b-414d-8abe-5afc0cb4ea8c None admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:50.039467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee140cc-1e7b-414d-8abe-5afc0cb4ea8c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:50.039813 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee140cc-1e7b-414d-8abe-5afc0cb4ea8c None admin] PATH_INFO: `/v3/projects/d6e147b6607b481591ceb250f7208c16/users/3bac5863e7bc4c7e854bd60912adb561/roles/70e0c42f1bdd49178db398637f021bbc` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:50.057314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee140cc-1e7b-414d-8abe-5afc0cb4ea8c None admin] RBAC: Authorizing `identity:create_grant(project_id=d6e147b6607b481591ceb250f7208c16, user_id=3bac5863e7bc4c7e854bd60912adb561, role_id=70e0c42f1bdd49178db398637f021bbc)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:50.058879 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee140cc-1e7b-414d-8abe-5afc0cb4ea8c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:50.073500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 618/1232] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:03:49 2022] PUT /identity/v3/projects/d6e147b6607b481591ceb250f7208c16/users/3bac5863e7bc4c7e854bd60912adb561/roles/70e0c42f1bdd49178db398637f021bbc => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:03:52.219711 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b8bfd-3dda-45ea-88c5-d1827b84be4d None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:52.220284 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b8bfd-3dda-45ea-88c5-d1827b84be4d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:52.220284 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b8bfd-3dda-45ea-88c5-d1827b84be4d None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:52.221605 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 615/1233] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:03:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:03:52.227784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723be2e5-cf14-4ed3-b989-49d411b5ac5c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:52.228291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723be2e5-cf14-4ed3-b989-49d411b5ac5c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:52.228291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723be2e5-cf14-4ed3-b989-49d411b5ac5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:52.252077 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-723be2e5-cf14-4ed3-b989-49d411b5ac5c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:52.253481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-723be2e5-cf14-4ed3-b989-49d411b5ac5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:52.332582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 619/1234] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:03:52 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:03:52.357754 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b8bfd-3dda-45ea-88c5-d1827b84be4d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:52.390903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:52.482384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b8bfd-3dda-45ea-88c5-d1827b84be4d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:52.521813 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4d1c7-967f-4e51-ab61-1c38e9948f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:52.522460 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4d1c7-967f-4e51-ab61-1c38e9948f5a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:52.522769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4d1c7-967f-4e51-ab61-1c38e9948f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:52.523046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a4d1c7-967f-4e51-ab61-1c38e9948f5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:52.535122 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a4d1c7-967f-4e51-ab61-1c38e9948f5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-93a4d1c7-967f-4e51-ab61-1c38e9948f5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:03:52.541641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:52.543542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:52.544543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:52.544543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:52.544543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:52.544543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:52.544543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:03:52.544543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:03:52.544543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:03:52.544543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:03:52.544543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:03:52.544543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:03:52.544543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 15:03:52.544543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:03:52.548145 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 616/1235] 10.222.0.100 () {56 vars in 1107 bytes} [Mon Aug 1 15:03:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:52.549762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723be2e5-cf14-4ed3-b989-49d411b5ac5c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:52.600029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:52.664830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723be2e5-cf14-4ed3-b989-49d411b5ac5c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:52.702475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f0f50-53b2-4fd6-9f91-3ba0f1151172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:52.703131 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f0f50-53b2-4fd6-9f91-3ba0f1151172 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:52.703411 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f0f50-53b2-4fd6-9f91-3ba0f1151172 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:52.703688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9f0f50-53b2-4fd6-9f91-3ba0f1151172 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:52.704406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f0f50-53b2-4fd6-9f91-3ba0f1151172 None admin] RBAC: Adding query filter params (name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:52.704712 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f0f50-53b2-4fd6-9f91-3ba0f1151172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:52.706082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9f0f50-53b2-4fd6-9f91-3ba0f1151172 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:52.714914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 620/1236] 10.222.0.100 () {56 vars in 1108 bytes} [Mon Aug 1 15:03:52 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:03:52.722305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4d1c7-967f-4e51-ab61-1c38e9948f5a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:52.811888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:52.830563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a4d1c7-967f-4e51-ab61-1c38e9948f5a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:52.868201 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d84bd-8a36-4bef-a0b5-a0cf6d68101d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:52.868919 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d84bd-8a36-4bef-a0b5-a0cf6d68101d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:52.869210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d84bd-8a36-4bef-a0b5-a0cf6d68101d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:52.869502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d84bd-8a36-4bef-a0b5-a0cf6d68101d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:52.881225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d84bd-8a36-4bef-a0b5-a0cf6d68101d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:52.882693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d84bd-8a36-4bef-a0b5-a0cf6d68101d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-6a8d84bd-8a36-4bef-a0b5-a0cf6d68101d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:03:52.897935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:52.899308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:52.900750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:03:52.900750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:03:52.900750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:52.900750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:03:52.900750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 617/1237] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:03:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:03:52.906110 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f0f50-53b2-4fd6-9f91-3ba0f1151172 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:53.026339 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9f0f50-53b2-4fd6-9f91-3ba0f1151172 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:53.040217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:53.070686 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681f37b-009f-4b4f-8d37-7bbcbb0cc1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:53.071280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681f37b-009f-4b4f-8d37-7bbcbb0cc1ae None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:53.071605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681f37b-009f-4b4f-8d37-7bbcbb0cc1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:53.071900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681f37b-009f-4b4f-8d37-7bbcbb0cc1ae None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:53.072630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681f37b-009f-4b4f-8d37-7bbcbb0cc1ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:53.072942 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681f37b-009f-4b4f-8d37-7bbcbb0cc1ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:53.074443 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681f37b-009f-4b4f-8d37-7bbcbb0cc1ae None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:53.087950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 621/1238] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:03:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:53.094511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d84bd-8a36-4bef-a0b5-a0cf6d68101d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:53.205632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d84bd-8a36-4bef-a0b5-a0cf6d68101d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:53.244100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db21e27-8304-489c-8082-d62c9f104064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:53.244753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db21e27-8304-489c-8082-d62c9f104064 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:53.245023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db21e27-8304-489c-8082-d62c9f104064 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:53.245304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db21e27-8304-489c-8082-d62c9f104064 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:53.248609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:53.254163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db21e27-8304-489c-8082-d62c9f104064 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:53.255950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db21e27-8304-489c-8082-d62c9f104064 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-4db21e27-8304-489c-8082-d62c9f104064 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:03:53.268294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:53.269444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:53.270404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:53.270404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:53.270404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:03:53.270404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:03:53.270404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:03:53.270404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:03:53.270404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:03:53.270404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:03:53.270404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:03:53.270404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:03:53.270404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 618/1239] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:03:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:53.275997 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681f37b-009f-4b4f-8d37-7bbcbb0cc1ae None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:53.381031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681f37b-009f-4b4f-8d37-7bbcbb0cc1ae None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:53.418714 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da3ba7-7c6b-4776-a3b7-1c3f4e4511f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:53.419408 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da3ba7-7c6b-4776-a3b7-1c3f4e4511f8 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:53.419692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da3ba7-7c6b-4776-a3b7-1c3f4e4511f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:53.419970 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da3ba7-7c6b-4776-a3b7-1c3f4e4511f8 None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:53.420763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da3ba7-7c6b-4776-a3b7-1c3f4e4511f8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:53.421076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da3ba7-7c6b-4776-a3b7-1c3f4e4511f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:53.422514 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da3ba7-7c6b-4776-a3b7-1c3f4e4511f8 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:53.443381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 622/1240] 10.222.0.100 () {56 vars in 1148 bytes} [Mon Aug 1 15:03:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:53.449745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db21e27-8304-489c-8082-d62c9f104064 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:53.569713 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db21e27-8304-489c-8082-d62c9f104064 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:53.611596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae82bef-38e6-4c93-a4ae-df450c834a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:53.612244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae82bef-38e6-4c93-a4ae-df450c834a32 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:53.612603 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae82bef-38e6-4c93-a4ae-df450c834a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:53.612881 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae82bef-38e6-4c93-a4ae-df450c834a32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:53.687438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:53.695873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae82bef-38e6-4c93-a4ae-df450c834a32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:53.697370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae82bef-38e6-4c93-a4ae-df450c834a32 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:53.813688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 619/1241] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:03:53 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:03:53.819703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da3ba7-7c6b-4776-a3b7-1c3f4e4511f8 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:53.851744 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:53.940372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da3ba7-7c6b-4776-a3b7-1c3f4e4511f8 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:53.983069 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c3f9a-18e4-4c49-bff1-53033d34a2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:53.983734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c3f9a-18e4-4c49-bff1-53033d34a2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:53.984045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c3f9a-18e4-4c49-bff1-53033d34a2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:53.984322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c3f9a-18e4-4c49-bff1-53033d34a2b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:53.995230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c3f9a-18e4-4c49-bff1-53033d34a2b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:53.996838 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c3f9a-18e4-4c49-bff1-53033d34a2b0 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-ef8c3f9a-18e4-4c49-bff1-53033d34a2b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:03:54.014718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:54.015889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:54.016893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:03:54.016893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:03:54.016893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:54.016893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:03:54.016893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 623/1242] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:54.021730 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae82bef-38e6-4c93-a4ae-df450c834a32 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:54.128448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:54.135216 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae82bef-38e6-4c93-a4ae-df450c834a32 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:54.175293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b29d8-1be4-48a7-ad7b-496bbea8d0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:54.176123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b29d8-1be4-48a7-ad7b-496bbea8d0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:54.176441 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b29d8-1be4-48a7-ad7b-496bbea8d0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:54.176810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b29d8-1be4-48a7-ad7b-496bbea8d0e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:54.177826 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b29d8-1be4-48a7-ad7b-496bbea8d0e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:54.178286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b29d8-1be4-48a7-ad7b-496bbea8d0e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:54.180080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b29d8-1be4-48a7-ad7b-496bbea8d0e7 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:54.193767 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 620/1243] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:03:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:03:54.199441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c3f9a-18e4-4c49-bff1-53033d34a2b0 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:54.292622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:54.317585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c3f9a-18e4-4c49-bff1-53033d34a2b0 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:54.354401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28797fe9-56af-45ee-be10-a7069c412023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:54.355080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28797fe9-56af-45ee-be10-a7069c412023 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:54.355364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28797fe9-56af-45ee-be10-a7069c412023 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:54.355640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28797fe9-56af-45ee-be10-a7069c412023 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:54.363979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28797fe9-56af-45ee-be10-a7069c412023 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:54.365638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28797fe9-56af-45ee-be10-a7069c412023 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-28797fe9-56af-45ee-be10-a7069c412023 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:03:54.378337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:54.379812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:54.380986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:03:54.380986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:03:54.380986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:03:54.380986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:03:54.380986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:03:54.380986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:03:54.380986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:03:54.380986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:03:54.380986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 624/1244] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:03:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:54.387836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b29d8-1be4-48a7-ad7b-496bbea8d0e7 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:54.503018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b29d8-1be4-48a7-ad7b-496bbea8d0e7 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:54.543267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ff3fe-6a9c-433d-8254-cfdd5477ff6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:54.543897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ff3fe-6a9c-433d-8254-cfdd5477ff6c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:54.544188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ff3fe-6a9c-433d-8254-cfdd5477ff6c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:54.544449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ff3fe-6a9c-433d-8254-cfdd5477ff6c None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:54.545208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ff3fe-6a9c-433d-8254-cfdd5477ff6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:54.545516 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ff3fe-6a9c-433d-8254-cfdd5477ff6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:54.547004 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ff3fe-6a9c-433d-8254-cfdd5477ff6c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:54.560105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 621/1245] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:03:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:54.567033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28797fe9-56af-45ee-be10-a7069c412023 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:54.680792 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28797fe9-56af-45ee-be10-a7069c412023 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:54.718535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39350c-91ea-4fdf-a1f7-ba7278b0c99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:54.719304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39350c-91ea-4fdf-a1f7-ba7278b0c99c None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:54.719592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39350c-91ea-4fdf-a1f7-ba7278b0c99c None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:54.719862 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39350c-91ea-4fdf-a1f7-ba7278b0c99c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:54.720664 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39350c-91ea-4fdf-a1f7-ba7278b0c99c None admin] RBAC: Adding query filter params (role.id=70e0c42f1bdd49178db398637f021bbc, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=3bac5863e7bc4c7e854bd60912adb561) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:54.720998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39350c-91ea-4fdf-a1f7-ba7278b0c99c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:54.722634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c39350c-91ea-4fdf-a1f7-ba7278b0c99c None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:54.729146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 625/1246] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:03:54 2022] GET /identity/v3/role_assignments?user.id=3bac5863e7bc4c7e854bd60912adb561&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=70e0c42f1bdd49178db398637f021bbc => generated 610 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:56.886367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502f701-437d-4e6e-8c6c-3d9db2c12dc7 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:56.886993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502f701-437d-4e6e-8c6c-3d9db2c12dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:56.886993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502f701-437d-4e6e-8c6c-3d9db2c12dc7 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:56.888915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 622/1247] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:03:56 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:03:56.895305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a9046-4f5a-4525-a9a7-0104a2c66de4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:56.895868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a9046-4f5a-4525-a9a7-0104a2c66de4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:56.896119 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a9046-4f5a-4525-a9a7-0104a2c66de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:56.903368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:56.917947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a36a9046-4f5a-4525-a9a7-0104a2c66de4 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:03:56.919287 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a36a9046-4f5a-4525-a9a7-0104a2c66de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:03:56.994563 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 626/1248] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:03:56 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:03:57.021631 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502f701-437d-4e6e-8c6c-3d9db2c12dc7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:57.044303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:57.147653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502f701-437d-4e6e-8c6c-3d9db2c12dc7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:57.186336 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caed7b-958d-4e26-b1aa-ef40d88c83d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:57.187019 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caed7b-958d-4e26-b1aa-ef40d88c83d6 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:57.187306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caed7b-958d-4e26-b1aa-ef40d88c83d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:57.187583 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94caed7b-958d-4e26-b1aa-ef40d88c83d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:57.204069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94caed7b-958d-4e26-b1aa-ef40d88c83d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-94caed7b-958d-4e26-b1aa-ef40d88c83d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:03:57.210352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:57.211471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:57.212480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:57.212480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:57.212480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:57.212480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:57.212480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:03:57.212480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:03:57.212480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:03:57.212480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:03:57.212480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:03:57.212480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:03:57.212480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:03:57.212480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:03:57.212480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 623/1249] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:03:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:57.220406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a9046-4f5a-4525-a9a7-0104a2c66de4 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:57.352017 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a9046-4f5a-4525-a9a7-0104a2c66de4 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:57.385578 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:57.391518 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd847ce5-9438-46fc-ba5e-61b267872c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:57.392224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd847ce5-9438-46fc-ba5e-61b267872c4b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:57.392514 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd847ce5-9438-46fc-ba5e-61b267872c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:57.392867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd847ce5-9438-46fc-ba5e-61b267872c4b None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:57.393594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd847ce5-9438-46fc-ba5e-61b267872c4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:57.393905 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd847ce5-9438-46fc-ba5e-61b267872c4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:57.395329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd847ce5-9438-46fc-ba5e-61b267872c4b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:57.405966 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 627/1250] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:03:57 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:57.412220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caed7b-958d-4e26-b1aa-ef40d88c83d6 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:57.508501 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:57.539539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94caed7b-958d-4e26-b1aa-ef40d88c83d6 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:57.588783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9d7a5-4776-4818-b1c2-2b8968e96437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:57.589521 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9d7a5-4776-4818-b1c2-2b8968e96437 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:57.589758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9d7a5-4776-4818-b1c2-2b8968e96437 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:57.590106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9d7a5-4776-4818-b1c2-2b8968e96437 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:57.602968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9d7a5-4776-4818-b1c2-2b8968e96437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:57.604459 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9d7a5-4776-4818-b1c2-2b8968e96437 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-98b9d7a5-4776-4818-b1c2-2b8968e96437 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:03:57.623411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:57.624720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:57.626269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:03:57.626269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:03:57.626269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:57.626269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:03:57.626269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 624/1251] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:03:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:57.631426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd847ce5-9438-46fc-ba5e-61b267872c4b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:57.759630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd847ce5-9438-46fc-ba5e-61b267872c4b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:57.803190 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e5acf-11a1-46f7-b819-69a626383871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:57.803988 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e5acf-11a1-46f7-b819-69a626383871 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:57.804384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e5acf-11a1-46f7-b819-69a626383871 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:57.804722 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e5acf-11a1-46f7-b819-69a626383871 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:57.805610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e5acf-11a1-46f7-b819-69a626383871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:57.806067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e5acf-11a1-46f7-b819-69a626383871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:57.807684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e5acf-11a1-46f7-b819-69a626383871 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:57.830598 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 628/1252] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:03:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:57.832266 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9d7a5-4776-4818-b1c2-2b8968e96437 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:57.956125 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9d7a5-4776-4818-b1c2-2b8968e96437 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:57.983891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:57.993160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50044349-757b-41ad-959b-9ab75ff4c05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:57.993850 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50044349-757b-41ad-959b-9ab75ff4c05b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:57.994251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50044349-757b-41ad-959b-9ab75ff4c05b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:57.994450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50044349-757b-41ad-959b-9ab75ff4c05b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:58.004551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50044349-757b-41ad-959b-9ab75ff4c05b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:58.006064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50044349-757b-41ad-959b-9ab75ff4c05b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-50044349-757b-41ad-959b-9ab75ff4c05b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:03:58.019672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:58.020710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:58.021759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:58.021759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:58.021759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:03:58.021759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:03:58.021759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:03:58.021759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:03:58.021759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:03:58.021759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:03:58.021759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:03:58.021759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:03:58.021759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 625/1253] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:03:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:58.026866 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e5acf-11a1-46f7-b819-69a626383871 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:58.091405 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:58.149057 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e5acf-11a1-46f7-b819-69a626383871 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:58.197565 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3233e-b9b7-468e-af3f-8ffd8949465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:58.198234 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3233e-b9b7-468e-af3f-8ffd8949465b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:58.198544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3233e-b9b7-468e-af3f-8ffd8949465b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:58.198880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3233e-b9b7-468e-af3f-8ffd8949465b None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:58.200485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3233e-b9b7-468e-af3f-8ffd8949465b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:58.200816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3233e-b9b7-468e-af3f-8ffd8949465b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:58.202452 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3233e-b9b7-468e-af3f-8ffd8949465b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:58.229862 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 629/1254] 10.222.0.100 () {56 vars in 1148 bytes} [Mon Aug 1 15:03:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:58.232434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50044349-757b-41ad-959b-9ab75ff4c05b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:58.363397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50044349-757b-41ad-959b-9ab75ff4c05b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:58.404256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735b8e5-55c7-4096-b8d6-e7641fe3239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:58.405002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735b8e5-55c7-4096-b8d6-e7641fe3239d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:58.405319 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735b8e5-55c7-4096-b8d6-e7641fe3239d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:58.405727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735b8e5-55c7-4096-b8d6-e7641fe3239d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:58.486563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:58.501748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735b8e5-55c7-4096-b8d6-e7641fe3239d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:58.503465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735b8e5-55c7-4096-b8d6-e7641fe3239d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:58.631836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 626/1255] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:03:58 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:03:58.641005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3233e-b9b7-468e-af3f-8ffd8949465b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:58.765884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3233e-b9b7-468e-af3f-8ffd8949465b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:58.781511 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:58.804535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffda2e-80f2-45ba-af69-9db755a3befe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:58.805253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffda2e-80f2-45ba-af69-9db755a3befe None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:58.805550 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffda2e-80f2-45ba-af69-9db755a3befe None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:58.805845 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffda2e-80f2-45ba-af69-9db755a3befe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:58.817707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffda2e-80f2-45ba-af69-9db755a3befe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:58.819453 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffda2e-80f2-45ba-af69-9db755a3befe None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-d3ffda2e-80f2-45ba-af69-9db755a3befe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:03:58.835596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:58.836688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:58.837773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:03:58.837773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:03:58.837773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:03:58.837773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:03:58.837773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 630/1256] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:58.843824 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735b8e5-55c7-4096-b8d6-e7641fe3239d None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:58.952996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:58.968461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735b8e5-55c7-4096-b8d6-e7641fe3239d None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:59.010105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc61d5-cf4c-49dd-9d9d-761122b5389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:59.011253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc61d5-cf4c-49dd-9d9d-761122b5389b None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:59.013255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc61d5-cf4c-49dd-9d9d-761122b5389b None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:59.013552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc61d5-cf4c-49dd-9d9d-761122b5389b None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:59.014272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc61d5-cf4c-49dd-9d9d-761122b5389b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:59.014579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc61d5-cf4c-49dd-9d9d-761122b5389b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:59.015993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc61d5-cf4c-49dd-9d9d-761122b5389b None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:59.027750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 627/1257] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:03:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:59.034148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffda2e-80f2-45ba-af69-9db755a3befe None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:59.145103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffda2e-80f2-45ba-af69-9db755a3befe None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:59.186489 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c1805-14ea-4319-bf60-d9b4c8bc28a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:59.187185 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c1805-14ea-4319-bf60-d9b4c8bc28a6 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:59.187468 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c1805-14ea-4319-bf60-d9b4c8bc28a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:59.187746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c1805-14ea-4319-bf60-d9b4c8bc28a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:59.196199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c1805-14ea-4319-bf60-d9b4c8bc28a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:59.197789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c1805-14ea-4319-bf60-d9b4c8bc28a6 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-8c7c1805-14ea-4319-bf60-d9b4c8bc28a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:03:59.219466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:03:59.220600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:03:59.221749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:03:59.221749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:03:59.221749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:03:59.221749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:03:59.221749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:03:59.221749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:03:59.221749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:03:59.221749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:03:59.222265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 631/1258] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:03:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:03:59.227721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc61d5-cf4c-49dd-9d9d-761122b5389b None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:59.358819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc61d5-cf4c-49dd-9d9d-761122b5389b None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:59.398153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76f96d-6105-4a09-b45c-2c8e6419d93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:59.398818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76f96d-6105-4a09-b45c-2c8e6419d93d None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:59.399103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76f96d-6105-4a09-b45c-2c8e6419d93d None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:59.399385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76f96d-6105-4a09-b45c-2c8e6419d93d None admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:59.400144 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76f96d-6105-4a09-b45c-2c8e6419d93d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:59.400507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76f96d-6105-4a09-b45c-2c8e6419d93d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:59.401912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76f96d-6105-4a09-b45c-2c8e6419d93d None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:59.415733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 628/1259] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:03:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:03:59.422404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c1805-14ea-4319-bf60-d9b4c8bc28a6 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:03:59.454046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:03:59.563635 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c1805-14ea-4319-bf60-d9b4c8bc28a6 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:03:59.608825 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570643dd-1181-4f81-a056-f0d1da410876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:03:59.609551 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570643dd-1181-4f81-a056-f0d1da410876 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:03:59.609876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570643dd-1181-4f81-a056-f0d1da410876 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:03:59.610156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570643dd-1181-4f81-a056-f0d1da410876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:03:59.610997 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570643dd-1181-4f81-a056-f0d1da410876 None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=3bac5863e7bc4c7e854bd60912adb561) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:03:59.611305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570643dd-1181-4f81-a056-f0d1da410876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:03:59.613010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570643dd-1181-4f81-a056-f0d1da410876 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:03:59.620177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 632/1260] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:03:59 2022] GET /identity/v3/role_assignments?user.id=3bac5863e7bc4c7e854bd60912adb561&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:01.687553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140cf29-9f3b-4733-a477-25317520b118 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:01.688123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140cf29-9f3b-4733-a477-25317520b118 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:01.688123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140cf29-9f3b-4733-a477-25317520b118 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:01.689463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 629/1261] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:04:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:04:01.694613 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef21b0-f9d3-475c-b639-f32d3b463243 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:01.695052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef21b0-f9d3-475c-b639-f32d3b463243 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:01.695200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef21b0-f9d3-475c-b639-f32d3b463243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:01.717328 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9cef21b0-f9d3-475c-b639-f32d3b463243 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:01.718576 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9cef21b0-f9d3-475c-b639-f32d3b463243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:01.784711 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 633/1262] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:04:01 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:04:01.807500 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140cf29-9f3b-4733-a477-25317520b118 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:01.819905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:01.916620 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140cf29-9f3b-4733-a477-25317520b118 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:01.950687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4359f-c735-4dbe-875b-3813c495f455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:01.951342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4359f-c735-4dbe-875b-3813c495f455 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:01.951626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4359f-c735-4dbe-875b-3813c495f455 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:01.951905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4359f-c735-4dbe-875b-3813c495f455 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:01.962128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4359f-c735-4dbe-875b-3813c495f455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-e2c4359f-c735-4dbe-875b-3813c495f455 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:04:01.967098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:04:01.968239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:04:01.969314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:04:01.969314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:04:01.969314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:04:01.969314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:04:01.969314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:04:01.969314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:04:01.969314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:04:01.969314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:04:01.969314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:04:01.969314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:04:01.969314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:04:01.969314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:04:01.969314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 630/1263] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:04:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:04:01.975772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef21b0-f9d3-475c-b639-f32d3b463243 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:01.994584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:02.087840 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef21b0-f9d3-475c-b639-f32d3b463243 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:02.123950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268a2bc-d39a-4604-93ea-4c4bdaa98af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:02.124707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268a2bc-d39a-4604-93ea-4c4bdaa98af2 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:02.124929 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268a2bc-d39a-4604-93ea-4c4bdaa98af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:02.125207 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268a2bc-d39a-4604-93ea-4c4bdaa98af2 None admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:02.126016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268a2bc-d39a-4604-93ea-4c4bdaa98af2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:02.126323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268a2bc-d39a-4604-93ea-4c4bdaa98af2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:02.127903 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268a2bc-d39a-4604-93ea-4c4bdaa98af2 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:02.138573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 634/1264] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:04:01 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:02.145704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4359f-c735-4dbe-875b-3813c495f455 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:02.215170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:02.249388 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4359f-c735-4dbe-875b-3813c495f455 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:02.286606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e43df-f38f-4c62-9073-04f48d7fd109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:02.287404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e43df-f38f-4c62-9073-04f48d7fd109 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:02.287790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e43df-f38f-4c62-9073-04f48d7fd109 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:02.288169 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e43df-f38f-4c62-9073-04f48d7fd109 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:02.298722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e43df-f38f-4c62-9073-04f48d7fd109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:02.300213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e43df-f38f-4c62-9073-04f48d7fd109 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-909e43df-f38f-4c62-9073-04f48d7fd109 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:04:02.316545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:04:02.318414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:04:02.320251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:04:02.320251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:04:02.320251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:04:02.320251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:04:02.320251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 631/1265] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:04:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:04:02.324579 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268a2bc-d39a-4604-93ea-4c4bdaa98af2 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:02.414531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:02.436850 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268a2bc-d39a-4604-93ea-4c4bdaa98af2 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:02.471898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dc3fd-5197-4479-9458-2bbc7f8855ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:02.472528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dc3fd-5197-4479-9458-2bbc7f8855ae None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:02.472812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dc3fd-5197-4479-9458-2bbc7f8855ae None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:02.473086 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dc3fd-5197-4479-9458-2bbc7f8855ae None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:02.473782 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dc3fd-5197-4479-9458-2bbc7f8855ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:02.474093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dc3fd-5197-4479-9458-2bbc7f8855ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:02.475504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dc3fd-5197-4479-9458-2bbc7f8855ae None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:02.488687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 635/1266] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:04:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:02.494743 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e43df-f38f-4c62-9073-04f48d7fd109 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:02.599234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e43df-f38f-4c62-9073-04f48d7fd109 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:02.619822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:02.634141 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb99f7b-41db-4b57-beff-fec371721bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:02.634976 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb99f7b-41db-4b57-beff-fec371721bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:02.635112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb99f7b-41db-4b57-beff-fec371721bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:02.635374 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb99f7b-41db-4b57-beff-fec371721bb4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:02.644120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb99f7b-41db-4b57-beff-fec371721bb4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:02.645575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb99f7b-41db-4b57-beff-fec371721bb4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-efb99f7b-41db-4b57-beff-fec371721bb4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:04:02.657539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:04:02.658735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:04:02.660102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:04:02.660102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:04:02.660102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:04:02.660102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:04:02.660102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:04:02.660102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:04:02.660102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:04:02.660102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:04:02.660102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:04:02.660102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:04:02.660102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 632/1267] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:04:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:04:02.664896 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dc3fd-5197-4479-9458-2bbc7f8855ae None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:02.769600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dc3fd-5197-4479-9458-2bbc7f8855ae None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:02.804091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a66f9-3f86-4f3e-923a-4c12e2465f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:02.804711 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a66f9-3f86-4f3e-923a-4c12e2465f8d None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:02.804990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a66f9-3f86-4f3e-923a-4c12e2465f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:02.805262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a66f9-3f86-4f3e-923a-4c12e2465f8d None admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:02.806045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a66f9-3f86-4f3e-923a-4c12e2465f8d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:02.806352 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a66f9-3f86-4f3e-923a-4c12e2465f8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:02.807920 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a66f9-3f86-4f3e-923a-4c12e2465f8d None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:02.816940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:02.826574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 636/1268] 10.222.0.100 () {56 vars in 1148 bytes} [Mon Aug 1 15:04:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:02.835895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb99f7b-41db-4b57-beff-fec371721bb4 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:02.942957 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb99f7b-41db-4b57-beff-fec371721bb4 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:02.978241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebfaa1-eaaf-4c45-9bb3-aa4c478a1b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:02.978972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebfaa1-eaaf-4c45-9bb3-aa4c478a1b75 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:02.979255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebfaa1-eaaf-4c45-9bb3-aa4c478a1b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:02.979530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebfaa1-eaaf-4c45-9bb3-aa4c478a1b75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:02.990262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebfaa1-eaaf-4c45-9bb3-aa4c478a1b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:02.991752 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebfaa1-eaaf-4c45-9bb3-aa4c478a1b75 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-f8ebfaa1-eaaf-4c45-9bb3-aa4c478a1b75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:04:03.006626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:04:03.007771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:04:03.014244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:04:03.014244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:04:03.014244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:04:03.014244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:04:03.014244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 633/1269] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:04:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:04:03.014600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a66f9-3f86-4f3e-923a-4c12e2465f8d None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:03.119617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a66f9-3f86-4f3e-923a-4c12e2465f8d None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:03.161897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3fa7a-9bfe-4110-a84d-3b80c6ccaa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:03.162530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3fa7a-9bfe-4110-a84d-3b80c6ccaa49 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:03.162847 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3fa7a-9bfe-4110-a84d-3b80c6ccaa49 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:03.163132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3fa7a-9bfe-4110-a84d-3b80c6ccaa49 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:03.163832 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3fa7a-9bfe-4110-a84d-3b80c6ccaa49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:03.164142 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3fa7a-9bfe-4110-a84d-3b80c6ccaa49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:03.165558 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3fa7a-9bfe-4110-a84d-3b80c6ccaa49 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:03.177435 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 637/1270] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:04:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:03.183291 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebfaa1-eaaf-4c45-9bb3-aa4c478a1b75 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:03.216959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:03.286898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebfaa1-eaaf-4c45-9bb3-aa4c478a1b75 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:03.320604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800c8de-cbe4-480f-ad7e-a3082d47b741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:03.321242 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800c8de-cbe4-480f-ad7e-a3082d47b741 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:03.321523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800c8de-cbe4-480f-ad7e-a3082d47b741 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:03.321835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800c8de-cbe4-480f-ad7e-a3082d47b741 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:03.329005 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800c8de-cbe4-480f-ad7e-a3082d47b741 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:03.330446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800c8de-cbe4-480f-ad7e-a3082d47b741 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-8800c8de-cbe4-480f-ad7e-a3082d47b741 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:04:03.341960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:04:03.343044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:04:03.344329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:04:03.344329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:04:03.344329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:04:03.344329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:04:03.344329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:04:03.344329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:04:03.344329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:04:03.344329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:04:03.344329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 634/1271] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:04:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:04:03.349070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3fa7a-9bfe-4110-a84d-3b80c6ccaa49 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:03.408290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:03.451459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3fa7a-9bfe-4110-a84d-3b80c6ccaa49 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:03.486130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f2237-4031-4928-a92d-90f5d9ecec92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:03.486651 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f2237-4031-4928-a92d-90f5d9ecec92 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:03.486957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f2237-4031-4928-a92d-90f5d9ecec92 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:03.487226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f2237-4031-4928-a92d-90f5d9ecec92 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:03.487963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f2237-4031-4928-a92d-90f5d9ecec92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:03.488269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f2237-4031-4928-a92d-90f5d9ecec92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:03.489696 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f2237-4031-4928-a92d-90f5d9ecec92 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:03.502264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 638/1272] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:04:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:03.508486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800c8de-cbe4-480f-ad7e-a3082d47b741 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:03.602052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:03.612362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800c8de-cbe4-480f-ad7e-a3082d47b741 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:03.647747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb3ac0-edf1-446c-b118-fa2333311948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:03.648428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb3ac0-edf1-446c-b118-fa2333311948 None admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:03.648712 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb3ac0-edf1-446c-b118-fa2333311948 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:03.648991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb3ac0-edf1-446c-b118-fa2333311948 None admin] PATH_INFO: `/v3/projects/d6e147b6607b481591ceb250f7208c16/users/3bac5863e7bc4c7e854bd60912adb561/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:03.650976 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb3ac0-edf1-446c-b118-fa2333311948 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6e147b6607b481591ceb250f7208c16, user_id=3bac5863e7bc4c7e854bd60912adb561, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:03.652439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb3ac0-edf1-446c-b118-fa2333311948 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:03.665790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 635/1273] 10.222.0.100 () {58 vars in 1444 bytes} [Mon Aug 1 15:04:03 2022] PUT /identity/v3/projects/d6e147b6607b481591ceb250f7208c16/users/3bac5863e7bc4c7e854bd60912adb561/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:04:05.688882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf460d44-2126-42ee-99fb-40c77d1b881f None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:05.689407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf460d44-2126-42ee-99fb-40c77d1b881f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:05.689407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf460d44-2126-42ee-99fb-40c77d1b881f None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:05.690817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 639/1274] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:04:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:04:05.696002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cf684-50c6-460e-b2ab-78e67f60b0d3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:05.696393 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cf684-50c6-460e-b2ab-78e67f60b0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:05.696569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cf684-50c6-460e-b2ab-78e67f60b0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:05.716849 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-872cf684-50c6-460e-b2ab-78e67f60b0d3 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:05.718056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-872cf684-50c6-460e-b2ab-78e67f60b0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:05.788178 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 636/1275] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:04:05 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:04:05.812310 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf460d44-2126-42ee-99fb-40c77d1b881f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:05.915649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf460d44-2126-42ee-99fb-40c77d1b881f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:05.933275 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:05.950500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e770d0-a534-4bbf-bd71-a11ff1a064c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:05.951684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e770d0-a534-4bbf-bd71-a11ff1a064c4 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:05.951684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e770d0-a534-4bbf-bd71-a11ff1a064c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:05.951890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e770d0-a534-4bbf-bd71-a11ff1a064c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:05.962278 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e770d0-a534-4bbf-bd71-a11ff1a064c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-19e770d0-a534-4bbf-bd71-a11ff1a064c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 15:04:05.967235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:04:05.968365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:04:05.969265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:04:05.969265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:04:05.969265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:04:05.969265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:04:05.969265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 15:04:05.969265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 15:04:05.969265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 15:04:05.969265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 15:04:05.969265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 15:04:05.969265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 15:04:05.969265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 15:04:05.969265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:04:05.969265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 640/1276] 10.222.0.100 () {56 vars in 1101 bytes} [Mon Aug 1 15:04:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:04:05.974432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cf684-50c6-460e-b2ab-78e67f60b0d3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:06.058956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:06.078591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cf684-50c6-460e-b2ab-78e67f60b0d3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:06.111562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec0f58-f405-4cda-80e6-b2bc591ec97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:06.112328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec0f58-f405-4cda-80e6-b2bc591ec97a None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:06.112789 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec0f58-f405-4cda-80e6-b2bc591ec97a None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:06.113092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec0f58-f405-4cda-80e6-b2bc591ec97a None admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:06.114102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec0f58-f405-4cda-80e6-b2bc591ec97a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:06.114558 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec0f58-f405-4cda-80e6-b2bc591ec97a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:06.116510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ec0f58-f405-4cda-80e6-b2bc591ec97a None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:06.128681 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 637/1277] 10.222.0.100 () {56 vars in 1104 bytes} [Mon Aug 1 15:04:05 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:06.134523 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e770d0-a534-4bbf-bd71-a11ff1a064c4 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:06.239321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e770d0-a534-4bbf-bd71-a11ff1a064c4 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:06.274214 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecfad4-f7b0-46a5-b704-a98f3b3140d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:06.274889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecfad4-f7b0-46a5-b704-a98f3b3140d7 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:06.275178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecfad4-f7b0-46a5-b704-a98f3b3140d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:06.275465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ecfad4-f7b0-46a5-b704-a98f3b3140d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:06.285461 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ecfad4-f7b0-46a5-b704-a98f3b3140d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:06.286949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ecfad4-f7b0-46a5-b704-a98f3b3140d7 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-b4ecfad4-f7b0-46a5-b704-a98f3b3140d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:04:06.301431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:04:06.302376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:04:06.303457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:04:06.303457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:04:06.303457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:04:06.303457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:04:06.303457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 641/1278] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:04:06.308595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec0f58-f405-4cda-80e6-b2bc591ec97a None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:06.413384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec0f58-f405-4cda-80e6-b2bc591ec97a None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:06.456141 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a649e-e3a4-418b-a83f-92c0b7b79ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:06.456762 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a649e-e3a4-418b-a83f-92c0b7b79ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:06.457047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a649e-e3a4-418b-a83f-92c0b7b79ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:06.457326 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a649e-e3a4-418b-a83f-92c0b7b79ae3 None admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:06.458020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a649e-e3a4-418b-a83f-92c0b7b79ae3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:06.458372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a649e-e3a4-418b-a83f-92c0b7b79ae3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:06.460044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a649e-e3a4-418b-a83f-92c0b7b79ae3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:06.471801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 638/1279] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:04:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:06.477684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecfad4-f7b0-46a5-b704-a98f3b3140d7 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:06.509255 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:06.591401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ecfad4-f7b0-46a5-b704-a98f3b3140d7 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:06.627006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d824e2d-705d-4b59-9937-64a160fb8511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:06.627651 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d824e2d-705d-4b59-9937-64a160fb8511 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:06.627931 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d824e2d-705d-4b59-9937-64a160fb8511 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:06.628206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d824e2d-705d-4b59-9937-64a160fb8511 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:06.635943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d824e2d-705d-4b59-9937-64a160fb8511 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:06.637451 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d824e2d-705d-4b59-9937-64a160fb8511 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-3d824e2d-705d-4b59-9937-64a160fb8511 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 15:04:06.649692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:04:06.656276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:04:06.658210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:04:06.658210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:04:06.658210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 15:04:06.658210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 15:04:06.658210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 15:04:06.658210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 15:04:06.658210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 15:04:06.658210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 15:04:06.658210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 15:04:06.658210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:04:06.658210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 642/1280] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:04:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:04:06.659689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a649e-e3a4-418b-a83f-92c0b7b79ae3 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:06.664293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:06.769884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a649e-e3a4-418b-a83f-92c0b7b79ae3 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:06.806845 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38b86e-f792-4f31-a0c7-7d971dfc0e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:06.807465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38b86e-f792-4f31-a0c7-7d971dfc0e78 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:06.807746 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38b86e-f792-4f31-a0c7-7d971dfc0e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:06.808021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38b86e-f792-4f31-a0c7-7d971dfc0e78 None admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:06.808815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38b86e-f792-4f31-a0c7-7d971dfc0e78 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:06.809121 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38b86e-f792-4f31-a0c7-7d971dfc0e78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:06.811239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c38b86e-f792-4f31-a0c7-7d971dfc0e78 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:06.830889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 639/1281] 10.222.0.100 () {56 vars in 1148 bytes} [Mon Aug 1 15:04:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:06.837018 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d824e2d-705d-4b59-9937-64a160fb8511 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:06.929221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:06.939816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d824e2d-705d-4b59-9937-64a160fb8511 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:06.975540 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8db61c-8245-479b-8e1b-292d9f2ba75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:06.976155 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8db61c-8245-479b-8e1b-292d9f2ba75e None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:06.976432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8db61c-8245-479b-8e1b-292d9f2ba75e None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:06.976703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8db61c-8245-479b-8e1b-292d9f2ba75e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:07.045500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8db61c-8245-479b-8e1b-292d9f2ba75e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:07.046971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8db61c-8245-479b-8e1b-292d9f2ba75e None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:07.145693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:07.158973 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 643/1282] 10.222.0.100 () {58 vars in 1287 bytes} [Mon Aug 1 15:04:06 2022] GET /identity/v3/auth/tokens => generated 2113 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 15:04:07.164856 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38b86e-f792-4f31-a0c7-7d971dfc0e78 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:07.228738 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:07.269423 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38b86e-f792-4f31-a0c7-7d971dfc0e78 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:07.304396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321e3c8-7271-412a-8b15-e508fbc11c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:07.305037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321e3c8-7271-412a-8b15-e508fbc11c76 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:07.305323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321e3c8-7271-412a-8b15-e508fbc11c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:07.305600 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321e3c8-7271-412a-8b15-e508fbc11c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:07.315963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a321e3c8-7271-412a-8b15-e508fbc11c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:07.317424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a321e3c8-7271-412a-8b15-e508fbc11c76 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-a321e3c8-7271-412a-8b15-e508fbc11c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 15:04:07.332747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:04:07.333872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:04:07.335181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 15:04:07.335181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 15:04:07.335181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 15:04:07.335181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:04:07.335181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 640/1283] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:04:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:04:07.340443 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8db61c-8245-479b-8e1b-292d9f2ba75e None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:07.445495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8db61c-8245-479b-8e1b-292d9f2ba75e None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:07.480295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331023ac-71b9-4b36-a8b1-3e8772546d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:07.480826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331023ac-71b9-4b36-a8b1-3e8772546d22 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:07.481105 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331023ac-71b9-4b36-a8b1-3e8772546d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:07.481373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331023ac-71b9-4b36-a8b1-3e8772546d22 None admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:07.482061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331023ac-71b9-4b36-a8b1-3e8772546d22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:07.482362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331023ac-71b9-4b36-a8b1-3e8772546d22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:07.483760 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331023ac-71b9-4b36-a8b1-3e8772546d22 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:07.496157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 644/1284] 10.222.0.100 () {56 vars in 1114 bytes} [Mon Aug 1 15:04:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:07.502736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321e3c8-7271-412a-8b15-e508fbc11c76 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:07.604396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321e3c8-7271-412a-8b15-e508fbc11c76 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:07.618431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:07.640501 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215ddea-400c-4589-90d8-2deb0b5c9d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:07.641197 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ddea-400c-4589-90d8-2deb0b5c9d28 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:07.641476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ddea-400c-4589-90d8-2deb0b5c9d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:07.641749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ddea-400c-4589-90d8-2deb0b5c9d28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:07.649416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215ddea-400c-4589-90d8-2deb0b5c9d28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:07.650923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215ddea-400c-4589-90d8-2deb0b5c9d28 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application [None req-9215ddea-400c-4589-90d8-2deb0b5c9d28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 15:04:07.662175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application with Lock( Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:04:07.663252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:04:07.664186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 15:04:07.664186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 15:04:07.664186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 15:04:07.664186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 15:04:07.664186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 15:04:07.664186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 15:04:07.664186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 15:04:07.664186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: ERROR keystone.server.flask.application Aug 01 15:04:07.664186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 641/1285] 10.222.0.100 () {56 vars in 1116 bytes} [Mon Aug 1 15:04:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:04:07.669332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331023ac-71b9-4b36-a8b1-3e8772546d22 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:07.719999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:07.771447 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331023ac-71b9-4b36-a8b1-3e8772546d22 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:07.805126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec9cfb-4d3d-48b4-9a49-79746a066e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:07.805742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9cfb-4d3d-48b4-9a49-79746a066e53 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:07.806031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9cfb-4d3d-48b4-9a49-79746a066e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:07.806312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9cfb-4d3d-48b4-9a49-79746a066e53 None admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:07.807420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec9cfb-4d3d-48b4-9a49-79746a066e53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:07.807420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec9cfb-4d3d-48b4-9a49-79746a066e53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:07.808847 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec9cfb-4d3d-48b4-9a49-79746a066e53 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:07.821611 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 645/1286] 10.222.0.100 () {56 vars in 1153 bytes} [Mon Aug 1 15:04:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:07.828393 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215ddea-400c-4589-90d8-2deb0b5c9d28 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:07.930080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215ddea-400c-4589-90d8-2deb0b5c9d28 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:07.963713 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5bf40-3f7c-4b0e-88a9-cf0e8bf87bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:07.964329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5bf40-3f7c-4b0e-88a9-cf0e8bf87bba None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:07.964616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5bf40-3f7c-4b0e-88a9-cf0e8bf87bba None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:07.964898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5bf40-3f7c-4b0e-88a9-cf0e8bf87bba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:07.965718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5bf40-3f7c-4b0e-88a9-cf0e8bf87bba None admin] RBAC: Adding query filter params (role.id=f18a6c6595d849f7a012306d67d167e5, scope.project.id=d6e147b6607b481591ceb250f7208c16, user.id=3bac5863e7bc4c7e854bd60912adb561) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:07.966033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5bf40-3f7c-4b0e-88a9-cf0e8bf87bba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:07.967504 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5bf40-3f7c-4b0e-88a9-cf0e8bf87bba None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:07.973439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 642/1287] 10.222.0.100 () {56 vars in 1379 bytes} [Mon Aug 1 15:04:07 2022] GET /identity/v3/role_assignments?user.id=3bac5863e7bc4c7e854bd60912adb561&scope.project.id=d6e147b6607b481591ceb250f7208c16&role.id=f18a6c6595d849f7a012306d67d167e5 => generated 610 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:10.042281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b235b2-57f1-4fd1-a837-c53d3b9e0b5d None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:10.042883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b235b2-57f1-4fd1-a837-c53d3b9e0b5d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:10.042883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b235b2-57f1-4fd1-a837-c53d3b9e0b5d None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:10.045025 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 646/1288] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:04:10 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:04:10.050080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735287-e1cf-45bd-bb45-ff6a0df744d3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:10.050522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735287-e1cf-45bd-bb45-ff6a0df744d3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:10.050613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735287-e1cf-45bd-bb45-ff6a0df744d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:10.068757 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-46735287-e1cf-45bd-bb45-ff6a0df744d3 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:10.069955 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-46735287-e1cf-45bd-bb45-ff6a0df744d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:10.100623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:10.146197 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 643/1289] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:04:10 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:04:10.173983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b235b2-57f1-4fd1-a837-c53d3b9e0b5d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:10.292775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b235b2-57f1-4fd1-a837-c53d3b9e0b5d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:10.304342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:10.330520 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9fd93-de17-4201-bfde-24889f39546b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:10.331213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9fd93-de17-4201-bfde-24889f39546b None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:10.331495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9fd93-de17-4201-bfde-24889f39546b None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:10.331771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9fd93-de17-4201-bfde-24889f39546b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:10.332403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9fd93-de17-4201-bfde-24889f39546b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:10.333795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9fd93-de17-4201-bfde-24889f39546b None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-8bf9fd93-de17-4201-bfde-24889f39546b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:04:10.343764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:04:10.344928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:04:10.346083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 647/1290] 10.222.0.100 () {56 vars in 1122 bytes} [Mon Aug 1 15:04:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:04:10.351838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46735287-e1cf-45bd-bb45-ff6a0df744d3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:10.453367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46735287-e1cf-45bd-bb45-ff6a0df744d3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:10.490881 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f6cc5-53a2-484b-80eb-1648a5c33c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:10.491499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f6cc5-53a2-484b-80eb-1648a5c33c6c None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:10.491780 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f6cc5-53a2-484b-80eb-1648a5c33c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:10.492053 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f6cc5-53a2-484b-80eb-1648a5c33c6c None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:10.492731 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f6cc5-53a2-484b-80eb-1648a5c33c6c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:10.493035 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f6cc5-53a2-484b-80eb-1648a5c33c6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:10.494420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f6cc5-53a2-484b-80eb-1648a5c33c6c None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:10.500104 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 644/1291] 10.222.0.100 () {56 vars in 1121 bytes} [Mon Aug 1 15:04:10 2022] GET /identity/v3/services?name=placement => generated 127 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:04:10.506171 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9fd93-de17-4201-bfde-24889f39546b None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:10.638135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9fd93-de17-4201-bfde-24889f39546b None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:10.689575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a6926e-2ba0-40df-812f-1a5af05b4cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:10.690510 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6926e-2ba0-40df-812f-1a5af05b4cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:10.690990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6926e-2ba0-40df-812f-1a5af05b4cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:10.691424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a6926e-2ba0-40df-812f-1a5af05b4cb3 None admin] PATH_INFO: `/v3/services` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:10.692506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6926e-2ba0-40df-812f-1a5af05b4cb3 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:10.693003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6926e-2ba0-40df-812f-1a5af05b4cb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:10.695500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a6926e-2ba0-40df-812f-1a5af05b4cb3 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:10.702569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 648/1292] 10.222.0.100 () {56 vars in 1121 bytes} [Mon Aug 1 15:04:10 2022] GET /identity/v3/services?type=placement => generated 127 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:12.728288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f393b61-36e9-4e27-9505-6846ba1de0b7 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:12.728850 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f393b61-36e9-4e27-9505-6846ba1de0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:12.728850 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f393b61-36e9-4e27-9505-6846ba1de0b7 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:12.730198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 645/1293] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:04:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:04:12.735973 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919a9f1-39de-4e9f-b00c-c7510cfd215b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:12.736516 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919a9f1-39de-4e9f-b00c-c7510cfd215b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:12.736716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919a9f1-39de-4e9f-b00c-c7510cfd215b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:12.755031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-6919a9f1-39de-4e9f-b00c-c7510cfd215b None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:12.756225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6919a9f1-39de-4e9f-b00c-c7510cfd215b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:12.785269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:12.826997 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 649/1294] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:04:12 2022] POST /identity/v3/auth/tokens => generated 2113 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:04:12.849638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f393b61-36e9-4e27-9505-6846ba1de0b7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:12.882571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:12.955078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f393b61-36e9-4e27-9505-6846ba1de0b7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:12.988870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8a3c9-673f-4e83-bebc-e7b68a6282c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:12.989494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8a3c9-673f-4e83-bebc-e7b68a6282c3 None admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:12.989779 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8a3c9-673f-4e83-bebc-e7b68a6282c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:12.990057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8a3c9-673f-4e83-bebc-e7b68a6282c3 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:12.990787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8a3c9-673f-4e83-bebc-e7b68a6282c3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:12.992212 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8a3c9-673f-4e83-bebc-e7b68a6282c3 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:13.007287 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 646/1295] 10.222.0.100 () {60 vars in 1146 bytes} [Mon Aug 1 15:04:12 2022] POST /identity/v3/services => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:04:15.065316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7713aa-9bfe-40bf-bdcf-ce98a510a56b None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:15.065976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7713aa-9bfe-40bf-bdcf-ce98a510a56b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:15.065976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7713aa-9bfe-40bf-bdcf-ce98a510a56b None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:15.067526 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 650/1296] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:04:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:04:15.072910 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b746e1-e417-4c77-8b6b-3e4eb38bfd29 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:15.073399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b746e1-e417-4c77-8b6b-3e4eb38bfd29 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:15.073505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b746e1-e417-4c77-8b6b-3e4eb38bfd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:15.092751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-23b746e1-e417-4c77-8b6b-3e4eb38bfd29 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:15.093965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-23b746e1-e417-4c77-8b6b-3e4eb38bfd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:15.191382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 647/1297] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:04:15 2022] POST /identity/v3/auth/tokens => generated 2216 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 15:04:15.225079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7713aa-9bfe-40bf-bdcf-ce98a510a56b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:15.341864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7713aa-9bfe-40bf-bdcf-ce98a510a56b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:15.379850 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9155829-327e-4164-8b6b-735d286fd532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:15.380489 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9155829-327e-4164-8b6b-735d286fd532 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:15.380799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9155829-327e-4164-8b6b-735d286fd532 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:15.381159 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9155829-327e-4164-8b6b-735d286fd532 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:15.381794 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9155829-327e-4164-8b6b-735d286fd532 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:15.383220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9155829-327e-4164-8b6b-735d286fd532 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-a9155829-327e-4164-8b6b-735d286fd532 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:04:15.393047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:04:15.394785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:04:15.396739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 651/1298] 10.222.0.100 () {56 vars in 1122 bytes} [Mon Aug 1 15:04:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 15:04:15.401241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b746e1-e417-4c77-8b6b-3e4eb38bfd29 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:15.429171 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:15.519749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b746e1-e417-4c77-8b6b-3e4eb38bfd29 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:15.558735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758c41e-c838-44fa-835c-e6c64e297a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:15.559943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758c41e-c838-44fa-835c-e6c64e297a61 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:15.560225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758c41e-c838-44fa-835c-e6c64e297a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:15.560501 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758c41e-c838-44fa-835c-e6c64e297a61 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:15.561183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758c41e-c838-44fa-835c-e6c64e297a61 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:15.561495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758c41e-c838-44fa-835c-e6c64e297a61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:15.562910 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758c41e-c838-44fa-835c-e6c64e297a61 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:15.568282 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 648/1299] 10.222.0.100 () {56 vars in 1121 bytes} [Mon Aug 1 15:04:15 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:15.574588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9155829-327e-4164-8b6b-735d286fd532 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:15.608955 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:15.685128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9155829-327e-4164-8b6b-735d286fd532 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:15.721792 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91572c77-ffa4-46c3-a46d-3d3a82263bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:15.722410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91572c77-ffa4-46c3-a46d-3d3a82263bea None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:15.722719 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91572c77-ffa4-46c3-a46d-3d3a82263bea None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:15.723001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91572c77-ffa4-46c3-a46d-3d3a82263bea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:15.723766 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91572c77-ffa4-46c3-a46d-3d3a82263bea None admin] RBAC: Adding query filter params (interface=public, service_id=cf408fb8e53645078e163dd4ab329c01, region_id=RegionOne) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:15.724076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91572c77-ffa4-46c3-a46d-3d3a82263bea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:15.725465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91572c77-ffa4-46c3-a46d-3d3a82263bea None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:15.731945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 652/1300] 10.222.0.100 () {56 vars in 1256 bytes} [Mon Aug 1 15:04:15 2022] GET /identity/v3/endpoints?service_id=cf408fb8e53645078e163dd4ab329c01&interface=public®ion_id=RegionOne => generated 195 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:15.737808 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758c41e-c838-44fa-835c-e6c64e297a61 None admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:15.827994 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:15.852340 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758c41e-c838-44fa-835c-e6c64e297a61 None admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:15.889215 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1f948-4bdd-4db7-89d4-8b6651464600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:15.889769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1f948-4bdd-4db7-89d4-8b6651464600 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:15.890050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1f948-4bdd-4db7-89d4-8b6651464600 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:15.890325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1f948-4bdd-4db7-89d4-8b6651464600 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:15.891003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1f948-4bdd-4db7-89d4-8b6651464600 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:15.892374 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1f948-4bdd-4db7-89d4-8b6651464600 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:15.899241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 649/1301] 10.222.0.100 () {56 vars in 1092 bytes} [Mon Aug 1 15:04:15 2022] GET /identity/v3/services => generated 2201 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:04:17.945629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcc3ce-5c87-46df-a564-f6c51a416ce7 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:17.946469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcc3ce-5c87-46df-a564-f6c51a416ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:17.946469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdcc3ce-5c87-46df-a564-f6c51a416ce7 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:17.947568 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 653/1302] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:04:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:04:17.954445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f3b38-0cf7-4345-b0b0-ad8842da0bee None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:17.954991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f3b38-0cf7-4345-b0b0-ad8842da0bee None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:17.955125 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f3b38-0cf7-4345-b0b0-ad8842da0bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:17.973417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-027f3b38-0cf7-4345-b0b0-ad8842da0bee None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:04:17.974635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-027f3b38-0cf7-4345-b0b0-ad8842da0bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:04:18.059583 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 650/1303] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:04:17 2022] POST /identity/v3/auth/tokens => generated 2216 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 15:04:18.082946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcc3ce-5c87-46df-a564-f6c51a416ce7 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:18.200248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:18.210820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdcc3ce-5c87-46df-a564-f6c51a416ce7 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:18.256473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb05a3c-b2c8-447b-8327-ade264139f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:18.257281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb05a3c-b2c8-447b-8327-ade264139f10 None admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:18.257638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb05a3c-b2c8-447b-8327-ade264139f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:18.258080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb05a3c-b2c8-447b-8327-ade264139f10 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:18.258943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb05a3c-b2c8-447b-8327-ade264139f10 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:18.260957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb05a3c-b2c8-447b-8327-ade264139f10 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application [None req-eeb05a3c-b2c8-447b-8327-ade264139f10 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 15:04:18.274442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 15:04:18.275790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application with Lock( Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self._enter() Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application return self.creator() Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application created_value = creator( Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: ERROR keystone.server.flask.application Aug 01 15:04:18.276855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 654/1304] 10.222.0.100 () {56 vars in 1122 bytes} [Mon Aug 1 15:04:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 15:04:18.284626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f3b38-0cf7-4345-b0b0-ad8842da0bee None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:18.392473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:18.422041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f3b38-0cf7-4345-b0b0-ad8842da0bee None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:18.458957 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d692d-521f-4cbd-9000-f5e1d74fecf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:18.459758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d692d-521f-4cbd-9000-f5e1d74fecf4 None admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:18.460035 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d692d-521f-4cbd-9000-f5e1d74fecf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:18.460305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d692d-521f-4cbd-9000-f5e1d74fecf4 None admin] PATH_INFO: `/v3/services` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:18.460988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d692d-521f-4cbd-9000-f5e1d74fecf4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:04:18.461292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d692d-521f-4cbd-9000-f5e1d74fecf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:18.462697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d692d-521f-4cbd-9000-f5e1d74fecf4 None admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:18.468212 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 651/1305] 10.222.0.100 () {56 vars in 1121 bytes} [Mon Aug 1 15:04:18 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:04:18.474161 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb05a3c-b2c8-447b-8327-ade264139f10 None admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:04:18.593205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb05a3c-b2c8-447b-8327-ade264139f10 None admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:04:18.628377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7971a5cc-fa55-4f90-99e7-3c0a8a6128d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:04:18.628997 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971a5cc-fa55-4f90-99e7-3c0a8a6128d4 None admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:04:18.629281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971a5cc-fa55-4f90-99e7-3c0a8a6128d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:04:18.629582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7971a5cc-fa55-4f90-99e7-3c0a8a6128d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:04:18.630271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971a5cc-fa55-4f90-99e7-3c0a8a6128d4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:04:18.631716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7971a5cc-fa55-4f90-99e7-3c0a8a6128d4 None admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:04:18.651835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 655/1306] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:04:18 2022] POST /identity/v3/endpoints => generated 333 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:04:28.464971 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=30 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:04:28.647084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:24.795743 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22139948-6375-4779-b0c5-c59d13eb34a5 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:24.796218 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22139948-6375-4779-b0c5-c59d13eb34a5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:24.796682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22139948-6375-4779-b0c5-c59d13eb34a5 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:24.806324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 652/1307] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:05:24 2022] GET /identity => generated 271 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:05:24.807617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ff684-05b2-413d-a618-256a8d3069a9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:24.808063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ff684-05b2-413d-a618-256a8d3069a9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:24.808232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ff684-05b2-413d-a618-256a8d3069a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:24.940625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-079ff684-05b2-413d-a618-256a8d3069a9 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:24.942795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-079ff684-05b2-413d-a618-256a8d3069a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:25.090889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 656/1308] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:05:25.168491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cac50-6484-43a3-bcad-c0d70c194f87 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:25.169100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cac50-6484-43a3-bcad-c0d70c194f87 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:25.169680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cac50-6484-43a3-bcad-c0d70c194f87 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:25.172672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 653/1309] 10.222.0.100 () {54 vars in 970 bytes} [Mon Aug 1 15:05:25 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:05:25.178940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9be75e-9ea1-4024-9fcb-dbe3b10e0a59 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:25.179305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9be75e-9ea1-4024-9fcb-dbe3b10e0a59 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:25.179631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9be75e-9ea1-4024-9fcb-dbe3b10e0a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:25.267730 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0a9be75e-9ea1-4024-9fcb-dbe3b10e0a59 None None] MFA Rules not processed for user `e7f757eabcb84de6ab042e0ac5a8158c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:25.269053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0a9be75e-9ea1-4024-9fcb-dbe3b10e0a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:25.293270 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:25.365462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 657/1310] 10.222.0.100 () {58 vars in 1067 bytes} [Mon Aug 1 15:05:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:05:25.376281 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cac50-6484-43a3-bcad-c0d70c194f87 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:25.378628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2f4cac50-6484-43a3-bcad-c0d70c194f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:25.506196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4cac50-6484-43a3-bcad-c0d70c194f87 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:25.622049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06ac0b-1548-4831-937d-96d552e8f64f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:25.622573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06ac0b-1548-4831-937d-96d552e8f64f service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:25.622889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06ac0b-1548-4831-937d-96d552e8f64f service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:25.623241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06ac0b-1548-4831-937d-96d552e8f64f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:25.624955 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2e06ac0b-1548-4831-937d-96d552e8f64f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:25.702255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:25.768615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06ac0b-1548-4831-937d-96d552e8f64f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:25.770089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06ac0b-1548-4831-937d-96d552e8f64f service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:25.791573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 654/1311] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:05:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:05:28.406143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ec0d2-d15e-4fbc-aa8b-f1d8fbab9982 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:28.406731 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ec0d2-d15e-4fbc-aa8b-f1d8fbab9982 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:28.406731 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ec0d2-d15e-4fbc-aa8b-f1d8fbab9982 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:28.408235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 658/1312] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:05:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:05:28.413780 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4cc6c-41bd-4bb1-8fba-4d15c1fd72b4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:28.415790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4cc6c-41bd-4bb1-8fba-4d15c1fd72b4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:28.415790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4cc6c-41bd-4bb1-8fba-4d15c1fd72b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:28.507670 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8aa4cc6c-41bd-4bb1-8fba-4d15c1fd72b4 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:28.509136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8aa4cc6c-41bd-4bb1-8fba-4d15c1fd72b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:28.620281 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 655/1313] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:05:28.659520 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ec0d2-d15e-4fbc-aa8b-f1d8fbab9982 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:28.661433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-846ec0d2-d15e-4fbc-aa8b-f1d8fbab9982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:28.813152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ec0d2-d15e-4fbc-aa8b-f1d8fbab9982 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:28.923156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:28.926318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa87cf-f5fc-4c14-896c-2d73e775cd30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:28.927241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa87cf-f5fc-4c14-896c-2d73e775cd30 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:28.927657 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa87cf-f5fc-4c14-896c-2d73e775cd30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:28.928140 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa87cf-f5fc-4c14-896c-2d73e775cd30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:28.930917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1efa87cf-f5fc-4c14-896c-2d73e775cd30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:29.076360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa87cf-f5fc-4c14-896c-2d73e775cd30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:29.077864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa87cf-f5fc-4c14-896c-2d73e775cd30 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:29.104280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 659/1314] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:05:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:31.499613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b8ef9-df1f-4a1b-8819-050f801eb336 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:31.500162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b8ef9-df1f-4a1b-8819-050f801eb336 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:31.500162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b8ef9-df1f-4a1b-8819-050f801eb336 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:31.501538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 656/1315] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:05:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:05:31.507073 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec4ffb-7fe6-4261-96db-f8a7a5737106 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:31.507539 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec4ffb-7fe6-4261-96db-f8a7a5737106 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:31.507660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec4ffb-7fe6-4261-96db-f8a7a5737106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:31.577038 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-48ec4ffb-7fe6-4261-96db-f8a7a5737106 None None] MFA Rules not processed for user `9d8a17c4135b4de4bf1d369d28f1c797`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:31.578289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-48ec4ffb-7fe6-4261-96db-f8a7a5737106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:31.634683 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:31.684322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 660/1316] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:05:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:31.738251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc19a1-9b4a-4340-811c-c4f65f9b4dfe None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:31.738594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc19a1-9b4a-4340-811c-c4f65f9b4dfe None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:31.738953 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc19a1-9b4a-4340-811c-c4f65f9b4dfe None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:31.740566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 657/1317] 10.222.0.100 () {54 vars in 970 bytes} [Mon Aug 1 15:05:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:05:31.748047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff80f82-e091-489a-91fa-19b65ff9f2e5 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:31.749608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff80f82-e091-489a-91fa-19b65ff9f2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:31.750086 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff80f82-e091-489a-91fa-19b65ff9f2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:31.872340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-dff80f82-e091-489a-91fa-19b65ff9f2e5 None None] MFA Rules not processed for user `e7f757eabcb84de6ab042e0ac5a8158c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:31.874179 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-dff80f82-e091-489a-91fa-19b65ff9f2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:31.991174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 661/1318] 10.222.0.100 () {58 vars in 1067 bytes} [Mon Aug 1 15:05:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:32.001408 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc19a1-9b4a-4340-811c-c4f65f9b4dfe None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:32.003004 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-62bc19a1-9b4a-4340-811c-c4f65f9b4dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:32.013067 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:32.127774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc19a1-9b4a-4340-811c-c4f65f9b4dfe None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:32.218263 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709efe0-c5c9-4772-8d0f-14dd608fd54a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:32.219211 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709efe0-c5c9-4772-8d0f-14dd608fd54a service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:32.219643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709efe0-c5c9-4772-8d0f-14dd608fd54a service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:32.220065 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709efe0-c5c9-4772-8d0f-14dd608fd54a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:32.222761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e709efe0-c5c9-4772-8d0f-14dd608fd54a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:32.295555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:32.356715 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709efe0-c5c9-4772-8d0f-14dd608fd54a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:32.358170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709efe0-c5c9-4772-8d0f-14dd608fd54a service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:32.377774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 658/1319] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:05:32 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:35.514727 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4742f-7f42-4537-883a-634159d79cbf None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:35.515294 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4742f-7f42-4537-883a-634159d79cbf None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:35.515294 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4742f-7f42-4537-883a-634159d79cbf None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:35.516666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 662/1320] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:05:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:05:35.522055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353513e3-1cea-4fe1-856c-b4a3e40d7a34 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:35.522515 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353513e3-1cea-4fe1-856c-b4a3e40d7a34 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:35.522784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353513e3-1cea-4fe1-856c-b4a3e40d7a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:35.595357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-353513e3-1cea-4fe1-856c-b4a3e40d7a34 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:35.596562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-353513e3-1cea-4fe1-856c-b4a3e40d7a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:35.704401 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 659/1321] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:05:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:35.744543 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4742f-7f42-4537-883a-634159d79cbf None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:35.746214 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-39e4742f-7f42-4537-883a-634159d79cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:35.805591 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:35.872531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4742f-7f42-4537-883a-634159d79cbf None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:35.953833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef492e5e-781b-4d74-ac29-962ed83ccb8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:35.954467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef492e5e-781b-4d74-ac29-962ed83ccb8b service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:35.954826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef492e5e-781b-4d74-ac29-962ed83ccb8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:35.955097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef492e5e-781b-4d74-ac29-962ed83ccb8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:35.956771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ef492e5e-781b-4d74-ac29-962ed83ccb8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:36.055916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:36.069515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef492e5e-781b-4d74-ac29-962ed83ccb8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:36.071568 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef492e5e-781b-4d74-ac29-962ed83ccb8b service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:36.089722 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 663/1322] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:05:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:39.515582 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6993e-31ba-44bc-a850-fa71f9e94237 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:39.516490 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6993e-31ba-44bc-a850-fa71f9e94237 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:39.516667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6993e-31ba-44bc-a850-fa71f9e94237 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:39.518425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 660/1323] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:05:39 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:05:39.525599 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd3148-33fe-432c-ae41-a6a3a84c900d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:39.526187 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd3148-33fe-432c-ae41-a6a3a84c900d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:39.526642 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd3148-33fe-432c-ae41-a6a3a84c900d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:39.614865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e5cd3148-33fe-432c-ae41-a6a3a84c900d None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:39.616236 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e5cd3148-33fe-432c-ae41-a6a3a84c900d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:39.732546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 664/1324] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:39.773824 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6993e-31ba-44bc-a850-fa71f9e94237 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:39.775521 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-daf6993e-31ba-44bc-a850-fa71f9e94237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:39.797222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:39.901667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6993e-31ba-44bc-a850-fa71f9e94237 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:40.002382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e986be1-3a08-459c-acb6-5f90fb6076f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:40.002963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e986be1-3a08-459c-acb6-5f90fb6076f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:40.003242 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e986be1-3a08-459c-acb6-5f90fb6076f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:40.003513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e986be1-3a08-459c-acb6-5f90fb6076f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:40.005168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1e986be1-3a08-459c-acb6-5f90fb6076f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:40.098761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:40.158026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e986be1-3a08-459c-acb6-5f90fb6076f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:40.159571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e986be1-3a08-459c-acb6-5f90fb6076f2 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:40.179684 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 661/1325] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:42.563536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736aa9a-2506-4263-bd95-ef24566616f7 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:42.564197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736aa9a-2506-4263-bd95-ef24566616f7 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:42.564197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736aa9a-2506-4263-bd95-ef24566616f7 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:42.565476 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 665/1326] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:05:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:05:42.571211 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a2974-3a95-4ab1-b257-effb1e9df376 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:42.572440 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a2974-3a95-4ab1-b257-effb1e9df376 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:42.572440 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a2974-3a95-4ab1-b257-effb1e9df376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:42.646273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f25a2974-3a95-4ab1-b257-effb1e9df376 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:42.647543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f25a2974-3a95-4ab1-b257-effb1e9df376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:42.767610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 662/1327] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:42.807796 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736aa9a-2506-4263-bd95-ef24566616f7 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:42.810358 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4736aa9a-2506-4263-bd95-ef24566616f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:42.880043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:42.954913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736aa9a-2506-4263-bd95-ef24566616f7 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:43.051828 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6c3ed-02b7-4a3b-ad64-1f323a424ffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:43.052697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6c3ed-02b7-4a3b-ad64-1f323a424ffa service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:43.052976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6c3ed-02b7-4a3b-ad64-1f323a424ffa service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:43.053248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6c3ed-02b7-4a3b-ad64-1f323a424ffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:43.055110 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5ab6c3ed-02b7-4a3b-ad64-1f323a424ffa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:43.182899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:43.187948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6c3ed-02b7-4a3b-ad64-1f323a424ffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:43.190738 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6c3ed-02b7-4a3b-ad64-1f323a424ffa service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:43.223240 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 666/1328] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:05:45.887853 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae4555-93fa-4031-81b7-9a17e4304327 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:45.888528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae4555-93fa-4031-81b7-9a17e4304327 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:45.888528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae4555-93fa-4031-81b7-9a17e4304327 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:45.890064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 663/1329] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:05:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:05:45.895654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965d7ff-7712-41c2-85b5-56eb45ecca44 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:45.896385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965d7ff-7712-41c2-85b5-56eb45ecca44 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:45.896626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965d7ff-7712-41c2-85b5-56eb45ecca44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:45.967710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7965d7ff-7712-41c2-85b5-56eb45ecca44 None None] MFA Rules not processed for user `9d8a17c4135b4de4bf1d369d28f1c797`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:45.968952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7965d7ff-7712-41c2-85b5-56eb45ecca44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:46.086790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 667/1330] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:05:46.123866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae4555-93fa-4031-81b7-9a17e4304327 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:46.126417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c9ae4555-93fa-4031-81b7-9a17e4304327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:46.168295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:46.269042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae4555-93fa-4031-81b7-9a17e4304327 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:46.349092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d947a1f-6ce7-452d-b291-40145639561a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:46.349716 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d947a1f-6ce7-452d-b291-40145639561a service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:46.349998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d947a1f-6ce7-452d-b291-40145639561a service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:46.350274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d947a1f-6ce7-452d-b291-40145639561a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:46.352009 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4d947a1f-6ce7-452d-b291-40145639561a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:46.440845 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:46.479458 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d947a1f-6ce7-452d-b291-40145639561a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:46.481061 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d947a1f-6ce7-452d-b291-40145639561a service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:46.502944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 664/1331] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:05:46 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:49.069379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d290b2-0ce4-4091-bdc1-6d5acdf8b195 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:49.069992 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d290b2-0ce4-4091-bdc1-6d5acdf8b195 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:49.069992 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d290b2-0ce4-4091-bdc1-6d5acdf8b195 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:49.071420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 668/1332] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:05:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:05:49.076747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f6f61-3d29-4085-a670-bc88fe66a9db None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:49.077220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f6f61-3d29-4085-a670-bc88fe66a9db None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:49.077343 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f6f61-3d29-4085-a670-bc88fe66a9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:49.149678 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3f8f6f61-3d29-4085-a670-bc88fe66a9db None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:49.150938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3f8f6f61-3d29-4085-a670-bc88fe66a9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:49.253100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 665/1333] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:49.289491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d290b2-0ce4-4091-bdc1-6d5acdf8b195 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:49.291254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-45d290b2-0ce4-4091-bdc1-6d5acdf8b195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:49.366232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:49.404404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d290b2-0ce4-4091-bdc1-6d5acdf8b195 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:49.486805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641d6e0-306f-43ae-bec0-3616d2ad1701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:49.487426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641d6e0-306f-43ae-bec0-3616d2ad1701 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:49.487711 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641d6e0-306f-43ae-bec0-3616d2ad1701 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:49.487985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641d6e0-306f-43ae-bec0-3616d2ad1701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:49.489663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9641d6e0-306f-43ae-bec0-3616d2ad1701 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:49.625425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641d6e0-306f-43ae-bec0-3616d2ad1701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:49.627087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641d6e0-306f-43ae-bec0-3616d2ad1701 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:49.638629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:49.648123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 669/1334] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:05:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:51.779090 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad1695-c64d-4a0a-b1df-7f5502566460 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:51.779834 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad1695-c64d-4a0a-b1df-7f5502566460 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:51.779834 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad1695-c64d-4a0a-b1df-7f5502566460 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:51.781190 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 666/1335] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:05:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:05:51.786935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce5810-527b-4751-a478-e8762f27ffa8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:51.787419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce5810-527b-4751-a478-e8762f27ffa8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:51.787561 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce5810-527b-4751-a478-e8762f27ffa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:51.858550 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-74ce5810-527b-4751-a478-e8762f27ffa8 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:51.859829 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-74ce5810-527b-4751-a478-e8762f27ffa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:51.966230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 670/1336] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:05:52.007690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad1695-c64d-4a0a-b1df-7f5502566460 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:52.009412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cbad1695-c64d-4a0a-b1df-7f5502566460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:52.052316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:52.126376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad1695-c64d-4a0a-b1df-7f5502566460 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:52.204076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d8fc4-4726-4930-b0a6-4ad9dcb6b260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:52.204841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d8fc4-4726-4930-b0a6-4ad9dcb6b260 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:52.205250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d8fc4-4726-4930-b0a6-4ad9dcb6b260 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:52.205611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d8fc4-4726-4930-b0a6-4ad9dcb6b260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:52.207697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-680d8fc4-4726-4930-b0a6-4ad9dcb6b260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:52.293329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:52.321199 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d8fc4-4726-4930-b0a6-4ad9dcb6b260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:52.323069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d8fc4-4726-4930-b0a6-4ad9dcb6b260 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:52.349450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 667/1337] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:05:54.967721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d24fc1-9566-42fc-9388-50307bee0b86 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:54.968255 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d24fc1-9566-42fc-9388-50307bee0b86 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:54.968567 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d24fc1-9566-42fc-9388-50307bee0b86 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:54.970913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 671/1338] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:05:54 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:05:54.976365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaaa31-0d7b-4a0c-9cd4-33576c8e0b78 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:54.976816 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaaa31-0d7b-4a0c-9cd4-33576c8e0b78 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:54.976992 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eaaa31-0d7b-4a0c-9cd4-33576c8e0b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:55.044398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-56eaaa31-0d7b-4a0c-9cd4-33576c8e0b78 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:05:55.045624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-56eaaa31-0d7b-4a0c-9cd4-33576c8e0b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:55.155686 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 668/1339] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:05:55.192210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d24fc1-9566-42fc-9388-50307bee0b86 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:05:55.193889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a2d24fc1-9566-42fc-9388-50307bee0b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:55.282970 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:05:55.312919 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d24fc1-9566-42fc-9388-50307bee0b86 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:05:55.389799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73beb647-a981-4b57-89e0-47b88adf8941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:05:55.390473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73beb647-a981-4b57-89e0-47b88adf8941 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:55.390893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73beb647-a981-4b57-89e0-47b88adf8941 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:55.391232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73beb647-a981-4b57-89e0-47b88adf8941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:55.392936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-73beb647-a981-4b57-89e0-47b88adf8941 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:05:55.507363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73beb647-a981-4b57-89e0-47b88adf8941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:05:55.508716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73beb647-a981-4b57-89e0-47b88adf8941 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:05:55.527473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 672/1340] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:05:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:05:59.933937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271a2b4-332e-4d05-801a-1b7fa799572a None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:59.934746 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271a2b4-332e-4d05-801a-1b7fa799572a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:59.934746 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271a2b4-332e-4d05-801a-1b7fa799572a None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:59.936213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 669/1341] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:05:59 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:05:59.942360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a208a-11cb-44b0-b3f4-6a28468339e6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:05:59.942867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a208a-11cb-44b0-b3f4-6a28468339e6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:05:59.942986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a208a-11cb-44b0-b3f4-6a28468339e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:05:59.949130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:00.017397 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f55a208a-11cb-44b0-b3f4-6a28468339e6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:00.018625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f55a208a-11cb-44b0-b3f4-6a28468339e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:00.136648 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 673/1342] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:00.199758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271a2b4-332e-4d05-801a-1b7fa799572a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:00.201627 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c271a2b4-332e-4d05-801a-1b7fa799572a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:00.255527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:00.323377 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271a2b4-332e-4d05-801a-1b7fa799572a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:00.411065 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e142b-52e9-45f8-bb63-eb7d6f1cfe22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:00.411613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e142b-52e9-45f8-bb63-eb7d6f1cfe22 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:00.411895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e142b-52e9-45f8-bb63-eb7d6f1cfe22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:00.412171 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e142b-52e9-45f8-bb63-eb7d6f1cfe22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:00.414390 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d43e142b-52e9-45f8-bb63-eb7d6f1cfe22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:00.519918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:00.544398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e142b-52e9-45f8-bb63-eb7d6f1cfe22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:00.545884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e142b-52e9-45f8-bb63-eb7d6f1cfe22 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:00.565640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 670/1343] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:03.330038 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1dffc-d150-48d4-9978-cb1116dce5a9 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:03.330618 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1dffc-d150-48d4-9978-cb1116dce5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:03.330618 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea1dffc-d150-48d4-9978-cb1116dce5a9 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:03.332078 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 674/1344] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:03.338891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86660d42-4ba2-4b55-9f4d-69a6135bb9eb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:03.339481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86660d42-4ba2-4b55-9f4d-69a6135bb9eb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:03.339481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86660d42-4ba2-4b55-9f4d-69a6135bb9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:03.441954 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-86660d42-4ba2-4b55-9f4d-69a6135bb9eb None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:03.443248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-86660d42-4ba2-4b55-9f4d-69a6135bb9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:03.565124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 671/1345] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:03.604408 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1dffc-d150-48d4-9978-cb1116dce5a9 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:03.606011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-dea1dffc-d150-48d4-9978-cb1116dce5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:03.693887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:03.713365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea1dffc-d150-48d4-9978-cb1116dce5a9 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:03.807125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0325bf-17aa-4369-9899-47372aa64875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:03.807908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0325bf-17aa-4369-9899-47372aa64875 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:03.808295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0325bf-17aa-4369-9899-47372aa64875 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:03.808664 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0325bf-17aa-4369-9899-47372aa64875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:03.811177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fc0325bf-17aa-4369-9899-47372aa64875 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:03.961680 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0325bf-17aa-4369-9899-47372aa64875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:03.964048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0325bf-17aa-4369-9899-47372aa64875 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:03.992695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 675/1346] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:08.185165 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c48967-3ab3-4d67-95a4-15d5f88a9e5b None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:08.185917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c48967-3ab3-4d67-95a4-15d5f88a9e5b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:08.186076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c48967-3ab3-4d67-95a4-15d5f88a9e5b None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:08.188626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 672/1347] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:08 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:08.196541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc5e28-0fa6-4290-bce7-20fcaa496be8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:08.197084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc5e28-0fa6-4290-bce7-20fcaa496be8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:08.197084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dc5e28-0fa6-4290-bce7-20fcaa496be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:08.215100 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:08.273231 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b6dc5e28-0fa6-4290-bce7-20fcaa496be8 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:08.274495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b6dc5e28-0fa6-4290-bce7-20fcaa496be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:08.433235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 676/1348] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:08.472999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c48967-3ab3-4d67-95a4-15d5f88a9e5b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:08.475102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-60c48967-3ab3-4d67-95a4-15d5f88a9e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:08.556670 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:08.638484 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c48967-3ab3-4d67-95a4-15d5f88a9e5b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:08.733762 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c74df5-39ee-404f-8f8c-4e3d39e564dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:08.734391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c74df5-39ee-404f-8f8c-4e3d39e564dc service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:08.734681 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c74df5-39ee-404f-8f8c-4e3d39e564dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:08.734975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c74df5-39ee-404f-8f8c-4e3d39e564dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:08.736664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-18c74df5-39ee-404f-8f8c-4e3d39e564dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:08.869718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:08.874507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c74df5-39ee-404f-8f8c-4e3d39e564dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:08.876055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c74df5-39ee-404f-8f8c-4e3d39e564dc service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:08.896593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 673/1349] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:14.203538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee8c49-bdf1-4512-bcb4-4c9069d4b71f None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:14.204092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee8c49-bdf1-4512-bcb4-4c9069d4b71f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:14.206701 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee8c49-bdf1-4512-bcb4-4c9069d4b71f None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:14.206701 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 677/1350] 10.222.0.100 () {54 vars in 926 bytes} [Mon Aug 1 15:06:14 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:14.215246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d43f1-d5d6-4904-af92-d831b97bcd85 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:14.215749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d43f1-d5d6-4904-af92-d831b97bcd85 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:14.215966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d43f1-d5d6-4904-af92-d831b97bcd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:14.332411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8e4d43f1-d5d6-4904-af92-d831b97bcd85 None None] MFA Rules not processed for user `3bac5863e7bc4c7e854bd60912adb561`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:14.333768 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8e4d43f1-d5d6-4904-af92-d831b97bcd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:14.487315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 674/1351] 10.222.0.100 () {58 vars in 1023 bytes} [Mon Aug 1 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:18.429574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=59 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:18.676707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaba45d-158b-4ace-9fe6-d776bddae599 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:18.677349 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaba45d-158b-4ace-9fe6-d776bddae599 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:18.678265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaba45d-158b-4ace-9fe6-d776bddae599 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:18.679801 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 678/1352] 10.222.0.100 () {54 vars in 926 bytes} [Mon Aug 1 15:06:18 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:18.685797 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4174ca-cd48-4946-9b56-1e805838de50 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:18.686323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4174ca-cd48-4946-9b56-1e805838de50 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:18.686323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4174ca-cd48-4946-9b56-1e805838de50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:18.747453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:18.774289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fe4174ca-cd48-4946-9b56-1e805838de50 None None] MFA Rules not processed for user `3bac5863e7bc4c7e854bd60912adb561`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:18.775559 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fe4174ca-cd48-4946-9b56-1e805838de50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:18.903100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 675/1353] 10.222.0.100 () {58 vars in 1023 bytes} [Mon Aug 1 15:06:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:20.391815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a611b2-b83d-4976-ac59-6654e8b9289d None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:20.392365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a611b2-b83d-4976-ac59-6654e8b9289d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:20.392365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a611b2-b83d-4976-ac59-6654e8b9289d None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:20.393710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 679/1354] 10.222.0.100 () {54 vars in 926 bytes} [Mon Aug 1 15:06:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:20.398949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c3aed-08e5-4bb9-b706-ea4dc386c08a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:20.399400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c3aed-08e5-4bb9-b706-ea4dc386c08a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:20.399515 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c3aed-08e5-4bb9-b706-ea4dc386c08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:20.480432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a25c3aed-08e5-4bb9-b706-ea4dc386c08a None None] MFA Rules not processed for user `3bac5863e7bc4c7e854bd60912adb561`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:20.481652 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a25c3aed-08e5-4bb9-b706-ea4dc386c08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:20.524060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:20.594750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 676/1355] 10.222.0.100 () {58 vars in 1023 bytes} [Mon Aug 1 15:06:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:24.092799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1591a8f-9c2a-4099-b194-625bbefba243 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:24.093360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1591a8f-9c2a-4099-b194-625bbefba243 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:24.093360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1591a8f-9c2a-4099-b194-625bbefba243 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:24.098714 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 680/1356] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:24.101149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579ccd6-bd8c-495a-a973-efeb9fb5ecb0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:24.101639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579ccd6-bd8c-495a-a973-efeb9fb5ecb0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:24.101753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579ccd6-bd8c-495a-a973-efeb9fb5ecb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:24.196915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1579ccd6-bd8c-495a-a973-efeb9fb5ecb0 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:24.198069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1579ccd6-bd8c-495a-a973-efeb9fb5ecb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:24.343266 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 677/1357] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:24.428261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70547d13-d3eb-4fd5-b1d3-43452464b7cc None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:24.428761 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70547d13-d3eb-4fd5-b1d3-43452464b7cc None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:24.428864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70547d13-d3eb-4fd5-b1d3-43452464b7cc None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:24.430296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 681/1358] 10.222.0.100 () {54 vars in 967 bytes} [Mon Aug 1 15:06:24 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:24.439430 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012832d7-dfe7-44ce-971b-36513649b92f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:24.439900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012832d7-dfe7-44ce-971b-36513649b92f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:24.440366 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012832d7-dfe7-44ce-971b-36513649b92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:24.469913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:24.587427 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-012832d7-dfe7-44ce-971b-36513649b92f None None] MFA Rules not processed for user `ec50ac7c54324884a1b2a6c63d617907`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:24.589617 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-012832d7-dfe7-44ce-971b-36513649b92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:24.737160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 678/1359] 10.222.0.100 () {58 vars in 1064 bytes} [Mon Aug 1 15:06:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:24.750564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70547d13-d3eb-4fd5-b1d3-43452464b7cc None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:24.752138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-70547d13-d3eb-4fd5-b1d3-43452464b7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:24.928267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70547d13-d3eb-4fd5-b1d3-43452464b7cc None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:25.006324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:25.048056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7974fc1-d5dc-454b-97b9-8ac9bd2fbd5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:25.048682 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7974fc1-d5dc-454b-97b9-8ac9bd2fbd5d service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:25.048966 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7974fc1-d5dc-454b-97b9-8ac9bd2fbd5d service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:25.049242 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7974fc1-d5dc-454b-97b9-8ac9bd2fbd5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:25.050885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d7974fc1-d5dc-454b-97b9-8ac9bd2fbd5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:25.148085 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7974fc1-d5dc-454b-97b9-8ac9bd2fbd5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:25.149692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7974fc1-d5dc-454b-97b9-8ac9bd2fbd5d service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:25.168794 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 682/1360] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:27.740007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a313f3a-7cd8-474b-aa6b-3ed67ffae028 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:27.740534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a313f3a-7cd8-474b-aa6b-3ed67ffae028 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:27.740648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a313f3a-7cd8-474b-aa6b-3ed67ffae028 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:27.742070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 679/1361] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:27.747189 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cf510-fa87-4245-9441-b84388534f3e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:27.747620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cf510-fa87-4245-9441-b84388534f3e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:27.747750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79cf510-fa87-4245-9441-b84388534f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:27.829363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d79cf510-fa87-4245-9441-b84388534f3e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:27.831549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d79cf510-fa87-4245-9441-b84388534f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:27.888027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:27.975676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 683/1362] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:28.020504 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a313f3a-7cd8-474b-aa6b-3ed67ffae028 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:28.022648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2a313f3a-7cd8-474b-aa6b-3ed67ffae028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:28.091550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:28.190782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a313f3a-7cd8-474b-aa6b-3ed67ffae028 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:28.338274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a2174-d5f9-4d58-b544-06ab48320cb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:28.339015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a2174-d5f9-4d58-b544-06ab48320cb0 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:28.339360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a2174-d5f9-4d58-b544-06ab48320cb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:28.339700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a2174-d5f9-4d58-b544-06ab48320cb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:28.342030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f17a2174-d5f9-4d58-b544-06ab48320cb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:28.369204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:28.460826 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a2174-d5f9-4d58-b544-06ab48320cb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:28.463230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a2174-d5f9-4d58-b544-06ab48320cb0 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:28.488398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 680/1363] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:30.879130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f73ee3-6cb5-4ece-aad6-5857daa0d290 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:30.879690 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f73ee3-6cb5-4ece-aad6-5857daa0d290 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:30.879690 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f73ee3-6cb5-4ece-aad6-5857daa0d290 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:30.881080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 684/1364] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:30.886426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd18b4d-ecb6-4c4e-89f7-2efb93eb864a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:30.886891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd18b4d-ecb6-4c4e-89f7-2efb93eb864a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:30.887784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd18b4d-ecb6-4c4e-89f7-2efb93eb864a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:30.958210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7cd18b4d-ecb6-4c4e-89f7-2efb93eb864a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:30.959465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7cd18b4d-ecb6-4c4e-89f7-2efb93eb864a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:31.067593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:31.070722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 681/1365] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:31.116589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d576cc-176a-41af-9f8e-c00a76c6159b None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:31.116880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d576cc-176a-41af-9f8e-c00a76c6159b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:31.117166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d576cc-176a-41af-9f8e-c00a76c6159b None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:31.118535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 685/1366] 10.222.0.100 () {54 vars in 967 bytes} [Mon Aug 1 15:06:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:31.124172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223deb9-4f1d-4d18-b9a4-5c67af0ee557 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:31.124465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223deb9-4f1d-4d18-b9a4-5c67af0ee557 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:31.124751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223deb9-4f1d-4d18-b9a4-5c67af0ee557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:31.205881 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e223deb9-4f1d-4d18-b9a4-5c67af0ee557 None None] MFA Rules not processed for user `ec50ac7c54324884a1b2a6c63d617907`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:31.207389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e223deb9-4f1d-4d18-b9a4-5c67af0ee557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:31.332803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 682/1367] 10.222.0.100 () {58 vars in 1064 bytes} [Mon Aug 1 15:06:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:31.341094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d576cc-176a-41af-9f8e-c00a76c6159b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:31.342623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-84d576cc-176a-41af-9f8e-c00a76c6159b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:31.510823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d576cc-176a-41af-9f8e-c00a76c6159b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:31.533110 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:31.665392 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56013970-347e-4222-aef1-33b92fd0d718 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:31.666137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56013970-347e-4222-aef1-33b92fd0d718 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:31.666476 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56013970-347e-4222-aef1-33b92fd0d718 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:31.666842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56013970-347e-4222-aef1-33b92fd0d718 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:31.668521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-56013970-347e-4222-aef1-33b92fd0d718 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:31.768718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56013970-347e-4222-aef1-33b92fd0d718 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:31.771245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56013970-347e-4222-aef1-33b92fd0d718 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:31.792725 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 686/1368] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:32.396457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b2c0-3854-423d-bd75-c129683d783e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:32.397046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b2c0-3854-423d-bd75-c129683d783e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:32.397046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b2c0-3854-423d-bd75-c129683d783e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:32.398405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 683/1369] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:32.403523 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f5c57-ffec-4d72-b8b7-760aeeafac30 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:32.404077 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f5c57-ffec-4d72-b8b7-760aeeafac30 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:32.404185 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f5c57-ffec-4d72-b8b7-760aeeafac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:32.438963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:32.477672 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f90f5c57-ffec-4d72-b8b7-760aeeafac30 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:32.478936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f90f5c57-ffec-4d72-b8b7-760aeeafac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:32.585240 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 687/1370] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:32.896591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd0641-6aa2-41ac-9cd8-a42b743e8e8c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:32.896970 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd0641-6aa2-41ac-9cd8-a42b743e8e8c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:32.897357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd0641-6aa2-41ac-9cd8-a42b743e8e8c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:32.898836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 684/1371] 10.222.0.100 () {54 vars in 969 bytes} [Mon Aug 1 15:06:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:32.903935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3f13f-1418-4b16-b2a1-0f291989a98b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:32.904227 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3f13f-1418-4b16-b2a1-0f291989a98b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:32.904509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3f13f-1418-4b16-b2a1-0f291989a98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:32.988290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e4f3f13f-1418-4b16-b2a1-0f291989a98b None None] MFA Rules not processed for user `ead1b36d35f3485f89edd15b8443feea`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:32.990102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e4f3f13f-1418-4b16-b2a1-0f291989a98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:33.043689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:33.078246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 688/1372] 10.222.0.100 () {58 vars in 1066 bytes} [Mon Aug 1 15:06:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:33.086001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd0641-6aa2-41ac-9cd8-a42b743e8e8c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:33.087587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9ebd0641-6aa2-41ac-9cd8-a42b743e8e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:33.147362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:33.211196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd0641-6aa2-41ac-9cd8-a42b743e8e8c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:33.304387 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd2871-54f5-41e9-9f4a-62e91df6e8bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:33.305118 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd2871-54f5-41e9-9f4a-62e91df6e8bc service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:33.305473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd2871-54f5-41e9-9f4a-62e91df6e8bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:33.305825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd2871-54f5-41e9-9f4a-62e91df6e8bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:33.307573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-34fd2871-54f5-41e9-9f4a-62e91df6e8bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:33.412090 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:33.426529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd2871-54f5-41e9-9f4a-62e91df6e8bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:33.428546 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd2871-54f5-41e9-9f4a-62e91df6e8bc service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:33.447618 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 685/1373] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:33.968157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8a144-a68e-48bd-8d71-2f59c83133ac None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:33.968799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8a144-a68e-48bd-8d71-2f59c83133ac None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:33.968799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8a144-a68e-48bd-8d71-2f59c83133ac None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:33.971114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 689/1374] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:33 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:33.975255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff30d69-f804-4e7f-86de-9797e94ec4c0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:33.975548 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff30d69-f804-4e7f-86de-9797e94ec4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:33.975829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff30d69-f804-4e7f-86de-9797e94ec4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:34.044355 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2ff30d69-f804-4e7f-86de-9797e94ec4c0 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:34.045587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2ff30d69-f804-4e7f-86de-9797e94ec4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:34.146654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 686/1375] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:34.194258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8a144-a68e-48bd-8d71-2f59c83133ac None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:34.195819 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bff8a144-a68e-48bd-8d71-2f59c83133ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:34.348137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff8a144-a68e-48bd-8d71-2f59c83133ac None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:34.402766 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:34.470008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79e5b7f-a19f-493f-8849-28ed83efaf61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:34.470549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e5b7f-a19f-493f-8849-28ed83efaf61 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:34.470870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e5b7f-a19f-493f-8849-28ed83efaf61 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:34.471148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79e5b7f-a19f-493f-8849-28ed83efaf61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:34.472797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b79e5b7f-a19f-493f-8849-28ed83efaf61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:34.566304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e5b7f-a19f-493f-8849-28ed83efaf61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:34.568466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79e5b7f-a19f-493f-8849-28ed83efaf61 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:34.587887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 690/1376] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:36.664235 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f86c0-14b2-462b-9455-93c38e01e2f0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:36.664730 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f86c0-14b2-462b-9455-93c38e01e2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:36.664730 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f86c0-14b2-462b-9455-93c38e01e2f0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:36.666055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 687/1377] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:36.671499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412cdac-23b1-4981-ab28-199ff4a5f334 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:36.671912 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412cdac-23b1-4981-ab28-199ff4a5f334 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:36.672065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412cdac-23b1-4981-ab28-199ff4a5f334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:36.740327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9412cdac-23b1-4981-ab28-199ff4a5f334 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:36.741544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9412cdac-23b1-4981-ab28-199ff4a5f334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:36.748188 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:36.841186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 691/1378] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:06:36.890904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f86c0-14b2-462b-9455-93c38e01e2f0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:36.892447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e85f86c0-14b2-462b-9455-93c38e01e2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:36.959397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:37.089524 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f86c0-14b2-462b-9455-93c38e01e2f0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:37.236293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d32972-9821-468e-91d0-45cdb882c6fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:37.236836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d32972-9821-468e-91d0-45cdb882c6fe service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:37.237115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d32972-9821-468e-91d0-45cdb882c6fe service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:37.237388 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d32972-9821-468e-91d0-45cdb882c6fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:37.239867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-85d32972-9821-468e-91d0-45cdb882c6fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:37.262813 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:37.348250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d32972-9821-468e-91d0-45cdb882c6fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:37.350059 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d32972-9821-468e-91d0-45cdb882c6fe service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:37.371092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 688/1379] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:39.450338 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f25dca-7005-4c07-80f5-b130ceebb1e8 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:39.450891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f25dca-7005-4c07-80f5-b130ceebb1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:39.450891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f25dca-7005-4c07-80f5-b130ceebb1e8 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:39.452434 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 692/1380] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:39.457727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0af5c6-6367-4bb2-b598-5e6998f6036d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:39.458832 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0af5c6-6367-4bb2-b598-5e6998f6036d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:39.458992 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0af5c6-6367-4bb2-b598-5e6998f6036d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:39.539270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6d0af5c6-6367-4bb2-b598-5e6998f6036d None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:39.539270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6d0af5c6-6367-4bb2-b598-5e6998f6036d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:39.641935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:39.649841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 689/1381] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:39.695405 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f25dca-7005-4c07-80f5-b130ceebb1e8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:39.696938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-37f25dca-7005-4c07-80f5-b130ceebb1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:39.849372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:39.861560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f25dca-7005-4c07-80f5-b130ceebb1e8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:40.026097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffe0f4-c398-4e41-8ff7-703d2fa9bff6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:40.027135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffe0f4-c398-4e41-8ff7-703d2fa9bff6 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:40.027495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffe0f4-c398-4e41-8ff7-703d2fa9bff6 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:40.027846 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffe0f4-c398-4e41-8ff7-703d2fa9bff6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:40.030122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-10ffe0f4-c398-4e41-8ff7-703d2fa9bff6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:40.129387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffe0f4-c398-4e41-8ff7-703d2fa9bff6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:40.131918 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ffe0f4-c398-4e41-8ff7-703d2fa9bff6 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:40.146353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:40.161967 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 693/1382] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:42.902548 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16016557-59fc-4794-82c3-d077afb8b315 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:42.903160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16016557-59fc-4794-82c3-d077afb8b315 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:42.903262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16016557-59fc-4794-82c3-d077afb8b315 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:42.904946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 690/1383] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:42 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:42.910684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36b234-45cc-422a-88f3-9fbf776b3ab9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:42.911677 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36b234-45cc-422a-88f3-9fbf776b3ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:42.911780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f36b234-45cc-422a-88f3-9fbf776b3ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:42.981867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5f36b234-45cc-422a-88f3-9fbf776b3ab9 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:42.983139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5f36b234-45cc-422a-88f3-9fbf776b3ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:43.092303 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 694/1384] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:43.141097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16016557-59fc-4794-82c3-d077afb8b315 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:43.143176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-16016557-59fc-4794-82c3-d077afb8b315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:43.305106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16016557-59fc-4794-82c3-d077afb8b315 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:43.396495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:43.457425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9bff1-d8c3-482d-a7ac-15c55fd8db5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:43.458286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9bff1-d8c3-482d-a7ac-15c55fd8db5e service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:43.458923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9bff1-d8c3-482d-a7ac-15c55fd8db5e service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:43.459275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9bff1-d8c3-482d-a7ac-15c55fd8db5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:43.461386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-31e9bff1-d8c3-482d-a7ac-15c55fd8db5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:43.581180 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9bff1-d8c3-482d-a7ac-15c55fd8db5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:43.583744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9bff1-d8c3-482d-a7ac-15c55fd8db5e service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:43.609910 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 691/1385] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:45.644566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d9691-1520-412d-90b4-c832c805670c None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:45.645214 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d9691-1520-412d-90b4-c832c805670c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:45.645214 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d9691-1520-412d-90b4-c832c805670c None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:45.646485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 695/1386] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:45.653310 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae9a48-42db-4b73-8db4-48b4fe907a77 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:45.653860 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae9a48-42db-4b73-8db4-48b4fe907a77 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:45.654349 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae9a48-42db-4b73-8db4-48b4fe907a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:45.734345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8bae9a48-42db-4b73-8db4-48b4fe907a77 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:45.735631 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8bae9a48-42db-4b73-8db4-48b4fe907a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:45.773208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:45.843753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 692/1387] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:45.888589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d9691-1520-412d-90b4-c832c805670c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:45.890132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-576d9691-1520-412d-90b4-c832c805670c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:45.978477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:46.055262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d9691-1520-412d-90b4-c832c805670c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:46.191469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9a36e-b154-4ba4-be13-75a6ec4a9f40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:46.192009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9a36e-b154-4ba4-be13-75a6ec4a9f40 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:46.192312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9a36e-b154-4ba4-be13-75a6ec4a9f40 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:46.192586 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9a36e-b154-4ba4-be13-75a6ec4a9f40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:46.194220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-39a9a36e-b154-4ba4-be13-75a6ec4a9f40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:46.235559 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:46.290253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9a36e-b154-4ba4-be13-75a6ec4a9f40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:46.291898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9a36e-b154-4ba4-be13-75a6ec4a9f40 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:46.311009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 696/1388] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:48.347384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696b221-a53d-4240-b584-ed67f3c6e563 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:48.347965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696b221-a53d-4240-b584-ed67f3c6e563 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:48.347965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696b221-a53d-4240-b584-ed67f3c6e563 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:48.349868 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 693/1389] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:48 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:48.355343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345c3c4-faaa-4a36-a0df-8108710887b9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:48.355759 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345c3c4-faaa-4a36-a0df-8108710887b9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:48.355935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345c3c4-faaa-4a36-a0df-8108710887b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:48.427005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7345c3c4-faaa-4a36-a0df-8108710887b9 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:48.428232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7345c3c4-faaa-4a36-a0df-8108710887b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:48.529511 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 697/1390] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:48.574419 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696b221-a53d-4240-b584-ed67f3c6e563 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:48.576211 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9696b221-a53d-4240-b584-ed67f3c6e563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:48.734420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696b221-a53d-4240-b584-ed67f3c6e563 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:48.752012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:48.863373 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa129b-8f4a-4b23-a320-8ea3f9ed3edf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:48.863990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa129b-8f4a-4b23-a320-8ea3f9ed3edf service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:48.864947 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa129b-8f4a-4b23-a320-8ea3f9ed3edf service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:48.865240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa129b-8f4a-4b23-a320-8ea3f9ed3edf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:48.866955 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-44aa129b-8f4a-4b23-a320-8ea3f9ed3edf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:48.956671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa129b-8f4a-4b23-a320-8ea3f9ed3edf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:48.958548 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa129b-8f4a-4b23-a320-8ea3f9ed3edf service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:48.977002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 694/1391] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:49.009804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d502a-a284-4afd-84de-fbc711c9a3e8 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:49.009804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d502a-a284-4afd-84de-fbc711c9a3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:49.009804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d502a-a284-4afd-84de-fbc711c9a3e8 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:49.009804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 698/1392] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:49 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:06:49.011594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89775f4-2f83-407f-8404-a1fdd899dc79 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:49.011885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89775f4-2f83-407f-8404-a1fdd899dc79 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:49.012167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89775f4-2f83-407f-8404-a1fdd899dc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:49.040264 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:49.088900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f89775f4-2f83-407f-8404-a1fdd899dc79 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:49.090138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f89775f4-2f83-407f-8404-a1fdd899dc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:49.208184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 695/1393] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:49.265028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f797788-3f8a-4ecc-8247-73aba7a1e0ff None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:49.265323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f797788-3f8a-4ecc-8247-73aba7a1e0ff None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:49.265609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f797788-3f8a-4ecc-8247-73aba7a1e0ff None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:49.267030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 699/1394] 10.222.0.100 () {54 vars in 971 bytes} [Mon Aug 1 15:06:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:49.272385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22b061-6d3b-4975-9361-bb492945714e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:49.272679 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22b061-6d3b-4975-9361-bb492945714e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:49.272962 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22b061-6d3b-4975-9361-bb492945714e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:49.351971 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9d22b061-6d3b-4975-9361-bb492945714e None None] MFA Rules not processed for user `e75edfc5bb404d63b27a355d5ab8af01`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:49.353195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9d22b061-6d3b-4975-9361-bb492945714e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:49.396182 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:49.432953 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 696/1395] 10.222.0.100 () {58 vars in 1068 bytes} [Mon Aug 1 15:06:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:49.440468 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f797788-3f8a-4ecc-8247-73aba7a1e0ff None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:49.442012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9f797788-3f8a-4ecc-8247-73aba7a1e0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:49.459528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:49.559232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f797788-3f8a-4ecc-8247-73aba7a1e0ff None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:49.637011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511879e-d27f-4a09-920b-b3117e5ff00c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e75edfc5bb404d63b27a355d5ab8af01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:49.637658 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511879e-d27f-4a09-920b-b3117e5ff00c service glance] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:49.637926 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511879e-d27f-4a09-920b-b3117e5ff00c service glance] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:49.638204 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511879e-d27f-4a09-920b-b3117e5ff00c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:49.639971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3511879e-d27f-4a09-920b-b3117e5ff00c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:49.715770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:49.765494 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511879e-d27f-4a09-920b-b3117e5ff00c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:49.766985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511879e-d27f-4a09-920b-b3117e5ff00c service glance] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:49.787169 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 700/1396] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:50.539850 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cd43a-27af-4b79-bcce-a950e374a97a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:50.540394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cd43a-27af-4b79-bcce-a950e374a97a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:50.540511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cd43a-27af-4b79-bcce-a950e374a97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:50.608565 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-145cd43a-27af-4b79-bcce-a950e374a97a None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:50.609801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-145cd43a-27af-4b79-bcce-a950e374a97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:50.704998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 697/1397] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:06:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:50.753315 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e8e49-4275-46a0-b806-3b0b615afc4a None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:50.753858 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e8e49-4275-46a0-b806-3b0b615afc4a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:50.753858 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e8e49-4275-46a0-b806-3b0b615afc4a None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:50.755369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 701/1398] 10.222.0.100 () {54 vars in 948 bytes} [Mon Aug 1 15:06:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:50.762014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bea5e-bf7a-4c5c-a5c4-aa93cb67eb15 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:50.762309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bea5e-bf7a-4c5c-a5c4-aa93cb67eb15 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:50.762594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bea5e-bf7a-4c5c-a5c4-aa93cb67eb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:50.836530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:50.844941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-411bea5e-bf7a-4c5c-a5c4-aa93cb67eb15 None None] MFA Rules not processed for user `41a399a3cc7446c6a8dad94075851797`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:50.846182 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-411bea5e-bf7a-4c5c-a5c4-aa93cb67eb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:50.955787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 698/1399] 10.222.0.100 () {58 vars in 1045 bytes} [Mon Aug 1 15:06:50 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:50.965464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e8e49-4275-46a0-b806-3b0b615afc4a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:50.967108 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0d8e8e49-4275-46a0-b806-3b0b615afc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:51.069088 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437317c-8cd0-4072-bbcd-ebeaddddadc1 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:51.069476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437317c-8cd0-4072-bbcd-ebeaddddadc1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:51.069833 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437317c-8cd0-4072-bbcd-ebeaddddadc1 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:51.074247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 699/1400] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:06:51.076577 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ae71d-45b3-4720-90d7-fdb9705ad123 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:51.076865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ae71d-45b3-4720-90d7-fdb9705ad123 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:51.077145 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ae71d-45b3-4720-90d7-fdb9705ad123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:51.107819 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e8e49-4275-46a0-b806-3b0b615afc4a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:51.146383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:51.148124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-830ae71d-45b3-4720-90d7-fdb9705ad123 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:51.149607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-830ae71d-45b3-4720-90d7-fdb9705ad123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:51.213376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:51.219957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c10c4-e78c-4417-9dcc-b3f6232d22ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:51.220570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c10c4-e78c-4417-9dcc-b3f6232d22ae service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:51.220850 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c10c4-e78c-4417-9dcc-b3f6232d22ae service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:51.221124 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c10c4-e78c-4417-9dcc-b3f6232d22ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:51.222882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5f0c10c4-e78c-4417-9dcc-b3f6232d22ae service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:51.251552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 700/1401] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:06:51.297534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ae71d-45b3-4720-90d7-fdb9705ad123 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:51.298776 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-830ae71d-45b3-4720-90d7-fdb9705ad123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:51.311280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c10c4-e78c-4417-9dcc-b3f6232d22ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:51.313105 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c10c4-e78c-4417-9dcc-b3f6232d22ae service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:51.321529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 702/1402] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:06:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:06:51.431917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ae71d-45b3-4720-90d7-fdb9705ad123 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:51.529606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:51.577728 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b57c78-3492-4ce0-bc38-ae05d86d4a2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:51.578428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b57c78-3492-4ce0-bc38-ae05d86d4a2d service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:51.578625 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b57c78-3492-4ce0-bc38-ae05d86d4a2d service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:51.578950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b57c78-3492-4ce0-bc38-ae05d86d4a2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:51.580774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-91b57c78-3492-4ce0-bc38-ae05d86d4a2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:51.680632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b57c78-3492-4ce0-bc38-ae05d86d4a2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:51.682349 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b57c78-3492-4ce0-bc38-ae05d86d4a2d service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:51.706855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 701/1403] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:06:53.839707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f030b6-3ec3-4e69-be8b-a65e561134ca None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:53.840284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f030b6-3ec3-4e69-be8b-a65e561134ca None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:53.840284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f030b6-3ec3-4e69-be8b-a65e561134ca None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:53.841678 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 703/1404] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:53.847550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c523b4-baa8-476a-bbda-9235c02191c6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:53.849137 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c523b4-baa8-476a-bbda-9235c02191c6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:53.849635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c523b4-baa8-476a-bbda-9235c02191c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:53.928948 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-51c523b4-baa8-476a-bbda-9235c02191c6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:53.930196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-51c523b4-baa8-476a-bbda-9235c02191c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:53.975564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:54.050814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 702/1405] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:54.094134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f030b6-3ec3-4e69-be8b-a65e561134ca None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:54.095766 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c0f030b6-3ec3-4e69-be8b-a65e561134ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:54.183982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:54.281128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f030b6-3ec3-4e69-be8b-a65e561134ca None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:54.431047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d530b-d0fd-43d5-aac9-7bede87d7251 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:54.431678 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d530b-d0fd-43d5-aac9-7bede87d7251 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:54.431959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d530b-d0fd-43d5-aac9-7bede87d7251 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:54.432232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d530b-d0fd-43d5-aac9-7bede87d7251 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:54.434000 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4f5d530b-d0fd-43d5-aac9-7bede87d7251 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:54.475360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:54.549602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d530b-d0fd-43d5-aac9-7bede87d7251 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:54.551341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d530b-d0fd-43d5-aac9-7bede87d7251 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:54.572399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 704/1406] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:06:56.659138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d0b78-c46a-4f87-8c34-22b5a0cac5a0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:56.659715 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d0b78-c46a-4f87-8c34-22b5a0cac5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:56.659715 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d0b78-c46a-4f87-8c34-22b5a0cac5a0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:56.661223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 703/1407] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:06:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:06:56.666357 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5836b2-2392-49df-9808-6c584b1052d0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:56.666857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5836b2-2392-49df-9808-6c584b1052d0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:56.667216 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5836b2-2392-49df-9808-6c584b1052d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:56.745445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ba5836b2-2392-49df-9808-6c584b1052d0 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:06:56.746867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ba5836b2-2392-49df-9808-6c584b1052d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:56.861391 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 705/1408] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:06:56.906049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d0b78-c46a-4f87-8c34-22b5a0cac5a0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:06:56.907632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-957d0b78-c46a-4f87-8c34-22b5a0cac5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:57.072638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d0b78-c46a-4f87-8c34-22b5a0cac5a0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:06:57.096828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:06:57.206784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027d22f-dfc3-487c-a13e-ee1d83e03f41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:06:57.206784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027d22f-dfc3-487c-a13e-ee1d83e03f41 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:06:57.207372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027d22f-dfc3-487c-a13e-ee1d83e03f41 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:06:57.207575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027d22f-dfc3-487c-a13e-ee1d83e03f41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:06:57.209850 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b027d22f-dfc3-487c-a13e-ee1d83e03f41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:06:57.351200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027d22f-dfc3-487c-a13e-ee1d83e03f41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:06:57.353406 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027d22f-dfc3-487c-a13e-ee1d83e03f41 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:06:57.390722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 704/1409] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:06.859203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:07.383358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:13.792170 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f1e82-e29a-4164-8e62-6c381508f803 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:13.792738 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f1e82-e29a-4164-8e62-6c381508f803 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:13.792738 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f1e82-e29a-4164-8e62-6c381508f803 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:13.794107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 706/1410] 10.222.0.100 () {54 vars in 923 bytes} [Mon Aug 1 15:07:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:13.799341 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67eb67-4d67-47cc-b931-36d5741ccbb2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:13.799783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67eb67-4d67-47cc-b931-36d5741ccbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:13.799882 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67eb67-4d67-47cc-b931-36d5741ccbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:13.876637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0d67eb67-4d67-47cc-b931-36d5741ccbb2 None None] MFA Rules not processed for user `3bac5863e7bc4c7e854bd60912adb561`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:13.877869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0d67eb67-4d67-47cc-b931-36d5741ccbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:13.987093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 705/1411] 10.222.0.100 () {58 vars in 1020 bytes} [Mon Aug 1 15:07:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:14.006644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f3920-2eec-4a71-baf3-0204d3d98c31 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:14.006961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f3920-2eec-4a71-baf3-0204d3d98c31 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:14.007240 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f3920-2eec-4a71-baf3-0204d3d98c31 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:14.008608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 707/1412] 10.222.0.100 () {54 vars in 923 bytes} [Mon Aug 1 15:07:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:14.013573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b32bfc-2b56-4614-b040-948282c32841 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:14.013863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b32bfc-2b56-4614-b040-948282c32841 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:14.014141 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b32bfc-2b56-4614-b040-948282c32841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:14.087937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:14.091662 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e0b32bfc-2b56-4614-b040-948282c32841 None None] MFA Rules not processed for user `ec50ac7c54324884a1b2a6c63d617907`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:14.093210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e0b32bfc-2b56-4614-b040-948282c32841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:14.209647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 706/1413] 10.222.0.100 () {58 vars in 1020 bytes} [Mon Aug 1 15:07:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:16.728491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1039a-1398-40c5-8320-e8f8856ce925 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:16.729063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1039a-1398-40c5-8320-e8f8856ce925 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:16.729063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1039a-1398-40c5-8320-e8f8856ce925 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:16.730417 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:07:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:16.736040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8ec88-461b-4a0b-b982-27c9df94145a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:16.736799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8ec88-461b-4a0b-b982-27c9df94145a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:16.736799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8ec88-461b-4a0b-b982-27c9df94145a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:16.832126 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7bf8ec88-461b-4a0b-b982-27c9df94145a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:16.833525 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7bf8ec88-461b-4a0b-b982-27c9df94145a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:16.890009 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:16.933448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:07:16.977879 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1039a-1398-40c5-8320-e8f8856ce925 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:16.979570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f9a1039a-1398-40c5-8320-e8f8856ce925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:17.138382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1039a-1398-40c5-8320-e8f8856ce925 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:17.215553 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:17.257660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a198c6-5445-4efb-a5ef-1f17945f9327 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:17.258276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a198c6-5445-4efb-a5ef-1f17945f9327 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:17.258552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a198c6-5445-4efb-a5ef-1f17945f9327 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:17.258854 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a198c6-5445-4efb-a5ef-1f17945f9327 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:17.260479 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d9a198c6-5445-4efb-a5ef-1f17945f9327 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:17.350439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a198c6-5445-4efb-a5ef-1f17945f9327 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:17.352159 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a198c6-5445-4efb-a5ef-1f17945f9327 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:17.370841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 709/1416] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:18.289280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2df52-787e-4d41-85e5-7a587a77f089 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:18.289857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2df52-787e-4d41-85e5-7a587a77f089 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:18.289857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2df52-787e-4d41-85e5-7a587a77f089 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:18.291213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:07:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:18.296444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549296e0-cb4a-4876-8be4-a4d4b1de5292 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:18.296890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549296e0-cb4a-4876-8be4-a4d4b1de5292 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:18.297085 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549296e0-cb4a-4876-8be4-a4d4b1de5292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:18.375861 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-549296e0-cb4a-4876-8be4-a4d4b1de5292 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:18.377091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-549296e0-cb4a-4876-8be4-a4d4b1de5292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:18.404624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:18.476500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:18.632168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2df52-787e-4d41-85e5-7a587a77f089 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:18.633761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-31e2df52-787e-4d41-85e5-7a587a77f089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:18.805985 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2df52-787e-4d41-85e5-7a587a77f089 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:18.852189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:18.927637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858daf2-11da-4ea9-804c-f87dda5d6e36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:18.928260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858daf2-11da-4ea9-804c-f87dda5d6e36 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:18.928541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858daf2-11da-4ea9-804c-f87dda5d6e36 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:18.928821 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858daf2-11da-4ea9-804c-f87dda5d6e36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:18.930465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7858daf2-11da-4ea9-804c-f87dda5d6e36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:19.022096 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858daf2-11da-4ea9-804c-f87dda5d6e36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:19.024277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7858daf2-11da-4ea9-804c-f87dda5d6e36 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:19.043449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 709/1419] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:07:19.939644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269696a-4fae-46f5-92dc-b82aec27598a None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:19.940198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269696a-4fae-46f5-92dc-b82aec27598a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:19.940198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269696a-4fae-46f5-92dc-b82aec27598a None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:19.941607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:07:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:19.946776 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739b17c-8f76-4426-b9ae-5dd82499f3ba None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:19.947484 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739b17c-8f76-4426-b9ae-5dd82499f3ba None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:19.947484 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739b17c-8f76-4426-b9ae-5dd82499f3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:20.006939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:20.030925 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a739b17c-8f76-4426-b9ae-5dd82499f3ba None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:20.032155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a739b17c-8f76-4426-b9ae-5dd82499f3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:20.133954 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:20.287557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269696a-4fae-46f5-92dc-b82aec27598a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:20.289261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0269696a-4fae-46f5-92dc-b82aec27598a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:20.439189 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269696a-4fae-46f5-92dc-b82aec27598a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:20.453227 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:20.558246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd1023-dc80-4a8d-9e53-eb252a1f2a4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:20.558891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd1023-dc80-4a8d-9e53-eb252a1f2a4f service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:20.559171 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd1023-dc80-4a8d-9e53-eb252a1f2a4f service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:20.559625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd1023-dc80-4a8d-9e53-eb252a1f2a4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:20.561377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4bcd1023-dc80-4a8d-9e53-eb252a1f2a4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:20.652898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd1023-dc80-4a8d-9e53-eb252a1f2a4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:20.654778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd1023-dc80-4a8d-9e53-eb252a1f2a4f service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:20.677414 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 712/1422] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:21.561398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfd315-2131-4503-9c0e-22a355c1b797 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:21.562148 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfd315-2131-4503-9c0e-22a355c1b797 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:21.562148 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfd315-2131-4503-9c0e-22a355c1b797 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:21.563360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:07:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:21.569131 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755028be-abb8-4f11-8f0e-8d42a7731a6f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:21.569570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755028be-abb8-4f11-8f0e-8d42a7731a6f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:21.569694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755028be-abb8-4f11-8f0e-8d42a7731a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:21.591436 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:21.657835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-755028be-abb8-4f11-8f0e-8d42a7731a6f None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:21.659130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-755028be-abb8-4f11-8f0e-8d42a7731a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:21.759993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:07:21.913339 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfd315-2131-4503-9c0e-22a355c1b797 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:21.914935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0fbfd315-2131-4503-9c0e-22a355c1b797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:22.028564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:22.064965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfd315-2131-4503-9c0e-22a355c1b797 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:22.181190 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0567040-4199-4706-a64c-c5dfdc6527e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:22.181720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0567040-4199-4706-a64c-c5dfdc6527e2 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:22.182005 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0567040-4199-4706-a64c-c5dfdc6527e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:22.182282 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0567040-4199-4706-a64c-c5dfdc6527e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:22.183956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a0567040-4199-4706-a64c-c5dfdc6527e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:22.264260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:22.279166 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0567040-4199-4706-a64c-c5dfdc6527e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:22.280770 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0567040-4199-4706-a64c-c5dfdc6527e2 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:22.299371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 712/1425] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:23.177646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ca7f7-1eb8-4881-98d3-61ca47803508 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:23.178172 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ca7f7-1eb8-4881-98d3-61ca47803508 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:23.178172 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ca7f7-1eb8-4881-98d3-61ca47803508 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:23.179555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:07:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:07:23.184430 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97391527-b67d-4ad7-aa39-ea9bb1976553 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:23.184862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97391527-b67d-4ad7-aa39-ea9bb1976553 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:23.184993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97391527-b67d-4ad7-aa39-ea9bb1976553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:23.263455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-97391527-b67d-4ad7-aa39-ea9bb1976553 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:23.264698 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-97391527-b67d-4ad7-aa39-ea9bb1976553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:23.366771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:23.519649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ca7f7-1eb8-4881-98d3-61ca47803508 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:23.521203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5f6ca7f7-1eb8-4881-98d3-61ca47803508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:23.614616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:23.679639 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ca7f7-1eb8-4881-98d3-61ca47803508 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:23.794625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a971185-3359-4ea9-a639-04b93fd9fc13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:23.795199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a971185-3359-4ea9-a639-04b93fd9fc13 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:23.795492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a971185-3359-4ea9-a639-04b93fd9fc13 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:23.795770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a971185-3359-4ea9-a639-04b93fd9fc13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:23.797409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1a971185-3359-4ea9-a639-04b93fd9fc13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:23.848884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:23.891485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a971185-3359-4ea9-a639-04b93fd9fc13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:23.893088 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a971185-3359-4ea9-a639-04b93fd9fc13 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:23.915915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 715/1428] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:24.739133 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb31f36-5060-470e-bba0-225479091cf0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:24.739740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb31f36-5060-470e-bba0-225479091cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:24.739740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb31f36-5060-470e-bba0-225479091cf0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:24.741964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:07:24 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:24.746964 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f895318-7b32-4630-95eb-e1ce63658888 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:24.747547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f895318-7b32-4630-95eb-e1ce63658888 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:24.747547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f895318-7b32-4630-95eb-e1ce63658888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:24.830007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-8f895318-7b32-4630-95eb-e1ce63658888 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:24.831305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8f895318-7b32-4630-95eb-e1ce63658888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:24.936058 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:25.089359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb31f36-5060-470e-bba0-225479091cf0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:25.090846 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dbb31f36-5060-470e-bba0-225479091cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:25.142468 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:25.240997 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb31f36-5060-470e-bba0-225479091cf0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:25.358354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb9b47-1073-4e31-aa33-584c9d0c8795 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:25.359011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb9b47-1073-4e31-aa33-584c9d0c8795 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:25.359290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb9b47-1073-4e31-aa33-584c9d0c8795 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:25.359563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb9b47-1073-4e31-aa33-584c9d0c8795 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:25.361191 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-80cb9b47-1073-4e31-aa33-584c9d0c8795 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:25.368293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:25.455844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb9b47-1073-4e31-aa33-584c9d0c8795 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:25.457477 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb9b47-1073-4e31-aa33-584c9d0c8795 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:25.477047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 715/1431] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:26.290268 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c37cee-4e43-4e92-96ac-aaa2af07ddd0 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:26.290774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c37cee-4e43-4e92-96ac-aaa2af07ddd0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:26.290891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c37cee-4e43-4e92-96ac-aaa2af07ddd0 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:26.292337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:07:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:26.297615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5ebf5-b388-4f4e-8de9-d9303169e102 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:26.298081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5ebf5-b388-4f4e-8de9-d9303169e102 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:26.298192 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5ebf5-b388-4f4e-8de9-d9303169e102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:26.377561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-baa5ebf5-b388-4f4e-8de9-d9303169e102 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:26.378819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-baa5ebf5-b388-4f4e-8de9-d9303169e102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:26.452994 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:26.487361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:26.642225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c37cee-4e43-4e92-96ac-aaa2af07ddd0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:26.643844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-62c37cee-4e43-4e92-96ac-aaa2af07ddd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:26.662732 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:26.794607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c37cee-4e43-4e92-96ac-aaa2af07ddd0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:26.913643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:26.941343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f6f2a-1665-4205-b45a-791b08ee2efa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:26.941967 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f6f2a-1665-4205-b45a-791b08ee2efa service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:26.942253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f6f2a-1665-4205-b45a-791b08ee2efa service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:26.942529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f6f2a-1665-4205-b45a-791b08ee2efa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:26.945260 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fb0f6f2a-1665-4205-b45a-791b08ee2efa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:27.041407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f6f2a-1665-4205-b45a-791b08ee2efa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:27.043165 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f6f2a-1665-4205-b45a-791b08ee2efa service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:27.064341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 718/1434] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:27.875932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16729f-c6d5-4a60-98ed-6fa5582ba361 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:27.876528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16729f-c6d5-4a60-98ed-6fa5582ba361 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:27.876528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16729f-c6d5-4a60-98ed-6fa5582ba361 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:27.877855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:07:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:27.882805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283797c-1fb2-4b3e-8ce0-c39b97612e41 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:27.883247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283797c-1fb2-4b3e-8ce0-c39b97612e41 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:27.883359 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283797c-1fb2-4b3e-8ce0-c39b97612e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:27.961556 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e283797c-1fb2-4b3e-8ce0-c39b97612e41 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:27.962811 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e283797c-1fb2-4b3e-8ce0-c39b97612e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:28.005215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:28.060753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:28.212454 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16729f-c6d5-4a60-98ed-6fa5582ba361 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:28.214051 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ae16729f-c6d5-4a60-98ed-6fa5582ba361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:28.357945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16729f-c6d5-4a60-98ed-6fa5582ba361 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:28.412391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:28.473755 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cef664-4879-42ad-bf6d-6328588eb43a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:28.474895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cef664-4879-42ad-bf6d-6328588eb43a service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:28.475147 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cef664-4879-42ad-bf6d-6328588eb43a service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:28.475423 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cef664-4879-42ad-bf6d-6328588eb43a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:28.477086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-55cef664-4879-42ad-bf6d-6328588eb43a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:28.569383 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cef664-4879-42ad-bf6d-6328588eb43a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:28.571074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cef664-4879-42ad-bf6d-6328588eb43a service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:28.590563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 718/1437] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:29.405079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b5d60-ca96-4a9b-ad25-f2d38404aa44 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:29.405629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b5d60-ca96-4a9b-ad25-f2d38404aa44 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:29.405629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b5d60-ca96-4a9b-ad25-f2d38404aa44 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:29.407046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:07:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:29.412052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279fd21-27da-44cc-aa8d-537d3529c2d6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:29.412507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279fd21-27da-44cc-aa8d-537d3529c2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:29.412644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c279fd21-27da-44cc-aa8d-537d3529c2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:29.488235 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c279fd21-27da-44cc-aa8d-537d3529c2d6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:29.489475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c279fd21-27da-44cc-aa8d-537d3529c2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:29.492033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:29.589305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:29.742410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b5d60-ca96-4a9b-ad25-f2d38404aa44 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:29.743903 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6b6b5d60-ca96-4a9b-ad25-f2d38404aa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:29.896027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b5d60-ca96-4a9b-ad25-f2d38404aa44 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:29.923439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:30.009625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc7c95-d16c-4025-b595-581c3c29ea7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:30.010157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc7c95-d16c-4025-b595-581c3c29ea7e service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:30.010435 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc7c95-d16c-4025-b595-581c3c29ea7e service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:30.010735 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc7c95-d16c-4025-b595-581c3c29ea7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:30.012351 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b0cc7c95-d16c-4025-b595-581c3c29ea7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:30.105750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc7c95-d16c-4025-b595-581c3c29ea7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:30.107416 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc7c95-d16c-4025-b595-581c3c29ea7e service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:30.126890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 721/1440] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:32.047095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b91497-9957-49a8-bb1e-121c1e7adab2 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:32.047671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b91497-9957-49a8-bb1e-121c1e7adab2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:32.047671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b91497-9957-49a8-bb1e-121c1e7adab2 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:32.049002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:07:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:32.054202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8706372-c1f6-46ea-9ffa-7184be2833a1 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:32.054639 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8706372-c1f6-46ea-9ffa-7184be2833a1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:32.054799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8706372-c1f6-46ea-9ffa-7184be2833a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:32.096867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:32.122295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d8706372-c1f6-46ea-9ffa-7184be2833a1 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:32.123548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d8706372-c1f6-46ea-9ffa-7184be2833a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:32.222138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:32.260711 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b91497-9957-49a8-bb1e-121c1e7adab2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:32.263244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-82b91497-9957-49a8-bb1e-121c1e7adab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:32.369399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b91497-9957-49a8-bb1e-121c1e7adab2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:32.401673 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:32.442958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c911f-f125-418c-ab6f-a4052a90269f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:32.443584 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c911f-f125-418c-ab6f-a4052a90269f service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:32.443868 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c911f-f125-418c-ab6f-a4052a90269f service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:32.444146 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c911f-f125-418c-ab6f-a4052a90269f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:32.445792 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-982c911f-f125-418c-ab6f-a4052a90269f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:32.559212 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c911f-f125-418c-ab6f-a4052a90269f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:32.560671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c911f-f125-418c-ab6f-a4052a90269f service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:32.578470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 721/1443] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:34.762234 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27ef0d-fd8a-4ccf-a361-212dbd70c891 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:34.762792 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27ef0d-fd8a-4ccf-a361-212dbd70c891 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:34.762908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27ef0d-fd8a-4ccf-a361-212dbd70c891 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:34.764206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:07:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:34.769453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be8cb7-21e7-4195-a5d2-2ef586e808c7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:34.769902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be8cb7-21e7-4195-a5d2-2ef586e808c7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:34.770010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be8cb7-21e7-4195-a5d2-2ef586e808c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:34.835617 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-35be8cb7-21e7-4195-a5d2-2ef586e808c7 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:34.836841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-35be8cb7-21e7-4195-a5d2-2ef586e808c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:34.849906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:34.934222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:07:34.973655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27ef0d-fd8a-4ccf-a361-212dbd70c891 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:34.975934 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ba27ef0d-fd8a-4ccf-a361-212dbd70c891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:35.077557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27ef0d-fd8a-4ccf-a361-212dbd70c891 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:35.090599 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:35.156034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aabb59-326d-4e7e-8cb1-bf40432d27a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:35.156654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aabb59-326d-4e7e-8cb1-bf40432d27a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:35.156944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aabb59-326d-4e7e-8cb1-bf40432d27a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:35.157218 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aabb59-326d-4e7e-8cb1-bf40432d27a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:35.158904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-28aabb59-326d-4e7e-8cb1-bf40432d27a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:35.268071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aabb59-326d-4e7e-8cb1-bf40432d27a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:35.269496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aabb59-326d-4e7e-8cb1-bf40432d27a3 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:35.288305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 724/1446] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:37.691547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05030c-6f28-4372-af2b-4f45f4e054ab None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:37.692131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05030c-6f28-4372-af2b-4f45f4e054ab None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:37.692131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05030c-6f28-4372-af2b-4f45f4e054ab None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:37.694022 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:07:37 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:37.699857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c418d51-d378-4b63-8921-9a6764b0db8b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:37.700386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c418d51-d378-4b63-8921-9a6764b0db8b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:37.700868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c418d51-d378-4b63-8921-9a6764b0db8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:37.751045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:37.770133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7c418d51-d378-4b63-8921-9a6764b0db8b None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:37.771387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7c418d51-d378-4b63-8921-9a6764b0db8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:37.868253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:07:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:07:37.906606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05030c-6f28-4372-af2b-4f45f4e054ab None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:37.908182 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ec05030c-6f28-4372-af2b-4f45f4e054ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:38.012758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05030c-6f28-4372-af2b-4f45f4e054ab None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:38.053410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:38.087569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc4940-676a-412e-8a5a-fb1855689bdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:38.088187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc4940-676a-412e-8a5a-fb1855689bdd service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:38.088472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc4940-676a-412e-8a5a-fb1855689bdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:38.088749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc4940-676a-412e-8a5a-fb1855689bdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:38.090445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2afc4940-676a-412e-8a5a-fb1855689bdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:38.197903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc4940-676a-412e-8a5a-fb1855689bdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:38.199300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc4940-676a-412e-8a5a-fb1855689bdd service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:38.223479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 724/1449] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:40.371247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406756be-4b4b-4065-a467-161f74099492 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:40.371797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406756be-4b4b-4065-a467-161f74099492 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:40.371797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406756be-4b4b-4065-a467-161f74099492 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:40.373246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:07:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:40.378563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20c5c-199b-4971-bdd5-71635048e4f7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:40.379020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20c5c-199b-4971-bdd5-71635048e4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:40.379135 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d20c5c-199b-4971-bdd5-71635048e4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:40.445229 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-56d20c5c-199b-4971-bdd5-71635048e4f7 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:40.446427 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-56d20c5c-199b-4971-bdd5-71635048e4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:40.467701 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:40.553266 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:40.592560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406756be-4b4b-4065-a467-161f74099492 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:40.594106 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-406756be-4b4b-4065-a467-161f74099492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:40.699419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406756be-4b4b-4065-a467-161f74099492 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:40.719491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:40.776700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26013953-4ccd-4750-a84c-12b399f54ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:40.777317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26013953-4ccd-4750-a84c-12b399f54ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:40.777604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26013953-4ccd-4750-a84c-12b399f54ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:40.777881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26013953-4ccd-4750-a84c-12b399f54ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:40.779570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-26013953-4ccd-4750-a84c-12b399f54ed3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:40.889498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26013953-4ccd-4750-a84c-12b399f54ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:40.890981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26013953-4ccd-4750-a84c-12b399f54ed3 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:40.909421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 727/1452] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:42.970643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8a000-0b59-4418-843e-42c485a00b67 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:42.971211 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8a000-0b59-4418-843e-42c485a00b67 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:42.971375 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8a000-0b59-4418-843e-42c485a00b67 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:42.972804 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:07:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:42.979382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304941cb-d43c-4681-ac5d-528d5c3516bf None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:42.979829 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304941cb-d43c-4681-ac5d-528d5c3516bf None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:42.979935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304941cb-d43c-4681-ac5d-528d5c3516bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:43.044151 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:43.046974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-304941cb-d43c-4681-ac5d-528d5c3516bf None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:43.048770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-304941cb-d43c-4681-ac5d-528d5c3516bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:43.147404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:43.184869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8a000-0b59-4418-843e-42c485a00b67 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:43.186435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2fc8a000-0b59-4418-843e-42c485a00b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:43.290569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8a000-0b59-4418-843e-42c485a00b67 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:43.340301 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:43.365873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d4b74-0f13-4b4d-9134-b4d754f4049c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:43.366551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d4b74-0f13-4b4d-9134-b4d754f4049c service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:43.366883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d4b74-0f13-4b4d-9134-b4d754f4049c service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:43.367161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d4b74-0f13-4b4d-9134-b4d754f4049c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:43.368802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8c4d4b74-0f13-4b4d-9134-b4d754f4049c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:43.606455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d4b74-0f13-4b4d-9134-b4d754f4049c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:43.607940 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4d4b74-0f13-4b4d-9134-b4d754f4049c service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:43.625777 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 727/1455] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:45.678176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db940a4-d8df-49e7-ae25-d1af6aafccef None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:45.678796 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db940a4-d8df-49e7-ae25-d1af6aafccef None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:45.678796 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db940a4-d8df-49e7-ae25-d1af6aafccef None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:45.680121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:07:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:45.685242 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8ad51-aac6-48fd-838e-74333f3b63f3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:45.685670 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8ad51-aac6-48fd-838e-74333f3b63f3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:45.685828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8ad51-aac6-48fd-838e-74333f3b63f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:45.751912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2da8ad51-aac6-48fd-838e-74333f3b63f3 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:45.753116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2da8ad51-aac6-48fd-838e-74333f3b63f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:45.781665 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:45.850678 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:45.887963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db940a4-d8df-49e7-ae25-d1af6aafccef None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:45.889485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0db940a4-d8df-49e7-ae25-d1af6aafccef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:45.994366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db940a4-d8df-49e7-ae25-d1af6aafccef None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:46.023232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:46.072434 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86463ef-811a-4442-a6fd-a4f7ea3ab4ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:46.073051 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86463ef-811a-4442-a6fd-a4f7ea3ab4ff service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:46.073335 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86463ef-811a-4442-a6fd-a4f7ea3ab4ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:46.073607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86463ef-811a-4442-a6fd-a4f7ea3ab4ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:46.075277 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b86463ef-811a-4442-a6fd-a4f7ea3ab4ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:46.188956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86463ef-811a-4442-a6fd-a4f7ea3ab4ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:46.190400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86463ef-811a-4442-a6fd-a4f7ea3ab4ff service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:46.209428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 730/1458] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:48.306221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b9cb2-3879-4ddf-aac0-189f2b2d6759 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:48.306851 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b9cb2-3879-4ddf-aac0-189f2b2d6759 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:48.306851 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b9cb2-3879-4ddf-aac0-189f2b2d6759 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:48.308162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:07:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:48.313418 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538f66d-721a-41cc-9604-3d8b9fea5b99 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:48.313854 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538f66d-721a-41cc-9604-3d8b9fea5b99 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:48.314009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538f66d-721a-41cc-9604-3d8b9fea5b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:48.380798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-8538f66d-721a-41cc-9604-3d8b9fea5b99 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:48.382041 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8538f66d-721a-41cc-9604-3d8b9fea5b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:48.390365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:48.481251 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:48.518904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b9cb2-3879-4ddf-aac0-189f2b2d6759 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:48.520445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e48b9cb2-3879-4ddf-aac0-189f2b2d6759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:48.623282 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b9cb2-3879-4ddf-aac0-189f2b2d6759 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:48.682951 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:48.702465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb3266-9c72-44ac-8b05-1ba9e77bbf68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:48.703118 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3266-9c72-44ac-8b05-1ba9e77bbf68 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:48.703424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3266-9c72-44ac-8b05-1ba9e77bbf68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:48.703703 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb3266-9c72-44ac-8b05-1ba9e77bbf68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:48.705402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9ddb3266-9c72-44ac-8b05-1ba9e77bbf68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:48.820359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb3266-9c72-44ac-8b05-1ba9e77bbf68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:48.821794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb3266-9c72-44ac-8b05-1ba9e77bbf68 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:48.841987 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 730/1461] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:50.915233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26948e11-4279-4c05-bf8c-0d541568ef06 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:50.915806 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26948e11-4279-4c05-bf8c-0d541568ef06 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:50.915806 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26948e11-4279-4c05-bf8c-0d541568ef06 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:50.917226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:07:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:07:50.922536 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e71825-b134-46f8-9c3e-7bfd54494d90 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:50.923046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e71825-b134-46f8-9c3e-7bfd54494d90 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:50.923187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e71825-b134-46f8-9c3e-7bfd54494d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:50.989199 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-32e71825-b134-46f8-9c3e-7bfd54494d90 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:50.990972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-32e71825-b134-46f8-9c3e-7bfd54494d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:51.028830 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:51.087998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:07:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:51.126372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26948e11-4279-4c05-bf8c-0d541568ef06 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:51.127978 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-26948e11-4279-4c05-bf8c-0d541568ef06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:51.234109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26948e11-4279-4c05-bf8c-0d541568ef06 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:51.271506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:51.307281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef769ec1-85e4-459d-9bb6-9bb1b2849350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:51.307897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef769ec1-85e4-459d-9bb6-9bb1b2849350 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:51.308180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef769ec1-85e4-459d-9bb6-9bb1b2849350 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:51.308462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef769ec1-85e4-459d-9bb6-9bb1b2849350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:51.310093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ef769ec1-85e4-459d-9bb6-9bb1b2849350 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:51.419946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef769ec1-85e4-459d-9bb6-9bb1b2849350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:51.421383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef769ec1-85e4-459d-9bb6-9bb1b2849350 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:51.567760 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 733/1464] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:53.634164 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e948cc-f846-42f7-9eb4-9a87cd19e9db None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:53.634788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e948cc-f846-42f7-9eb4-9a87cd19e9db None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:53.634788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e948cc-f846-42f7-9eb4-9a87cd19e9db None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:53.636117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:07:53 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:53.642224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b72afa-ac91-46cb-bebc-32ae035de811 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:53.642696 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b72afa-ac91-46cb-bebc-32ae035de811 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:53.642825 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b72afa-ac91-46cb-bebc-32ae035de811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:53.714250 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-25b72afa-ac91-46cb-bebc-32ae035de811 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:53.715533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-25b72afa-ac91-46cb-bebc-32ae035de811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:53.734604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:53.816103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:53.858858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e948cc-f846-42f7-9eb4-9a87cd19e9db None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:53.860403 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-56e948cc-f846-42f7-9eb4-9a87cd19e9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:53.965906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e948cc-f846-42f7-9eb4-9a87cd19e9db None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:54.038568 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:54.045703 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba050a04-b7a7-45c6-9d4a-d607982ef462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:54.046329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba050a04-b7a7-45c6-9d4a-d607982ef462 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:54.046613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba050a04-b7a7-45c6-9d4a-d607982ef462 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:54.046919 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba050a04-b7a7-45c6-9d4a-d607982ef462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:54.048560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ba050a04-b7a7-45c6-9d4a-d607982ef462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:54.167857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba050a04-b7a7-45c6-9d4a-d607982ef462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:54.169297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba050a04-b7a7-45c6-9d4a-d607982ef462 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:54.190975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 733/1467] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:07:56.245636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fc72f-1823-4ac2-99b9-d2e79a07a6a6 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:56.246268 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fc72f-1823-4ac2-99b9-d2e79a07a6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:56.246268 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fc72f-1823-4ac2-99b9-d2e79a07a6a6 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:56.247499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:07:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:07:56.252747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4bc7b-fabc-4893-9e37-32573d50ef97 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:56.253164 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4bc7b-fabc-4893-9e37-32573d50ef97 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:56.253322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4bc7b-fabc-4893-9e37-32573d50ef97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:56.324889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-19f4bc7b-fabc-4893-9e37-32573d50ef97 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:07:56.326111 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-19f4bc7b-fabc-4893-9e37-32573d50ef97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:56.373904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:56.423481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:07:56.460735 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fc72f-1823-4ac2-99b9-d2e79a07a6a6 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:07:56.462297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5e9fc72f-1823-4ac2-99b9-d2e79a07a6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:56.569838 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fc72f-1823-4ac2-99b9-d2e79a07a6a6 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:07:56.618523 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:07:56.644592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f913c-60cc-4e9d-82ec-cb72e3f46f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:07:56.645206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f913c-60cc-4e9d-82ec-cb72e3f46f66 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:07:56.645488 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f913c-60cc-4e9d-82ec-cb72e3f46f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:07:56.646335 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f913c-60cc-4e9d-82ec-cb72e3f46f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:07:56.648009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0a2f913c-60cc-4e9d-82ec-cb72e3f46f66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:07:56.756457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f913c-60cc-4e9d-82ec-cb72e3f46f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:07:56.757943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f913c-60cc-4e9d-82ec-cb72e3f46f66 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:07:56.776020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 736/1470] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:07:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:08:06.421613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=22 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:06.773820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=66 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:16.368274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb8e7e-a9c6-4bf2-bfba-454603cf8697 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:16.368838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb8e7e-a9c6-4bf2-bfba-454603cf8697 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:16.368838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb8e7e-a9c6-4bf2-bfba-454603cf8697 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:16.370172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 735/1471] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:08:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:08:16.375404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f5189-13b4-4317-b72a-a3e01fd1bcde None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:16.375862 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f5189-13b4-4317-b72a-a3e01fd1bcde None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:16.375983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f5189-13b4-4317-b72a-a3e01fd1bcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:16.441982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-608f5189-13b4-4317-b72a-a3e01fd1bcde None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:08:16.443826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-608f5189-13b4-4317-b72a-a3e01fd1bcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:16.539723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 737/1472] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:08:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:08:16.622962 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02f6c7-3c12-4e6f-819c-50e536623439 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:16.623199 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02f6c7-3c12-4e6f-819c-50e536623439 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:16.623482 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02f6c7-3c12-4e6f-819c-50e536623439 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:16.624821 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 736/1473] 10.222.0.100 () {54 vars in 971 bytes} [Mon Aug 1 15:08:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:08:16.629528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb180d-6bab-4afe-9dfa-bb6b93735ee9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:16.630357 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb180d-6bab-4afe-9dfa-bb6b93735ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:16.630634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb180d-6bab-4afe-9dfa-bb6b93735ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:16.704916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2bfb180d-6bab-4afe-9dfa-bb6b93735ee9 None None] MFA Rules not processed for user `e75edfc5bb404d63b27a355d5ab8af01`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:08:16.706127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2bfb180d-6bab-4afe-9dfa-bb6b93735ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:16.726286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:16.783101 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 738/1474] 10.222.0.100 () {58 vars in 1068 bytes} [Mon Aug 1 15:08:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:08:16.790690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02f6c7-3c12-4e6f-819c-50e536623439 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:16.792211 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fe02f6c7-3c12-4e6f-819c-50e536623439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:16.900745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02f6c7-3c12-4e6f-819c-50e536623439 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:16.973619 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac2f7d-b54d-4d4c-8289-102d75c969a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e75edfc5bb404d63b27a355d5ab8af01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:16.974652 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac2f7d-b54d-4d4c-8289-102d75c969a5 service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:16.974962 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac2f7d-b54d-4d4c-8289-102d75c969a5 service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:16.975236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac2f7d-b54d-4d4c-8289-102d75c969a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:16.976864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d3ac2f7d-b54d-4d4c-8289-102d75c969a5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:17.031251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:17.090134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac2f7d-b54d-4d4c-8289-102d75c969a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:17.091601 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac2f7d-b54d-4d4c-8289-102d75c969a5 service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:17.109622 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 737/1475] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:08:19.191697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07189ac0-bef9-4991-a553-2f069fe11a89 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:19.192884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07189ac0-bef9-4991-a553-2f069fe11a89 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:19.192884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07189ac0-bef9-4991-a553-2f069fe11a89 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:19.194148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 739/1476] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:08:19 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:08:19.199298 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c09d8-3a9f-4e58-9f93-c83ae4f93c6c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:19.199754 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c09d8-3a9f-4e58-9f93-c83ae4f93c6c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:19.199857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c09d8-3a9f-4e58-9f93-c83ae4f93c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:19.266447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-263c09d8-3a9f-4e58-9f93-c83ae4f93c6c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:08:19.267760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-263c09d8-3a9f-4e58-9f93-c83ae4f93c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:19.366291 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 738/1477] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:08:19.389572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07189ac0-bef9-4991-a553-2f069fe11a89 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:19.391148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-07189ac0-bef9-4991-a553-2f069fe11a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:19.519354 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07189ac0-bef9-4991-a553-2f069fe11a89 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:19.570227 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:19.616065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7519a-b70e-4d24-abf9-fe5c7705aea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:19.616673 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7519a-b70e-4d24-abf9-fe5c7705aea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:19.616955 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7519a-b70e-4d24-abf9-fe5c7705aea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:19.617227 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7519a-b70e-4d24-abf9-fe5c7705aea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:19.617918 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7519a-b70e-4d24-abf9-fe5c7705aea7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:19.619799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-3ba7519a-b70e-4d24-abf9-fe5c7705aea7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:08:19.619799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Aug 01 15:08:19.619799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: Aug 01 15:08:19.620941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba7519a-b70e-4d24-abf9-fe5c7705aea7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:19.639917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 740/1478] 10.222.0.100 () {56 vars in 1113 bytes} [Mon Aug 1 15:08:19 2022] GET /identity/v3/projects => generated 2562 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:08:21.668371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7fba7-8f33-4fe8-b4d8-55fa63d54a98 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:21.668934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7fba7-8f33-4fe8-b4d8-55fa63d54a98 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:21.668934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7fba7-8f33-4fe8-b4d8-55fa63d54a98 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:21.670264 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 739/1479] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:08:21 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:08:21.675387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9b57-6ee8-4578-9bb8-dee5d5369943 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:21.675830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9b57-6ee8-4578-9bb8-dee5d5369943 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:21.675938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee9b57-6ee8-4578-9bb8-dee5d5369943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:21.742831 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-62ee9b57-6ee8-4578-9bb8-dee5d5369943 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:08:21.744059 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-62ee9b57-6ee8-4578-9bb8-dee5d5369943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:21.839774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 741/1480] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:08:21.883637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7fba7-8f33-4fe8-b4d8-55fa63d54a98 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:21.885185 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-02d7fba7-8f33-4fe8-b4d8-55fa63d54a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:21.894820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:22.029901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7fba7-8f33-4fe8-b4d8-55fa63d54a98 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:22.113744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:22.148408 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e0a3e-b8d0-46d1-9823-280fb2a0a83b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:22.149025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e0a3e-b8d0-46d1-9823-280fb2a0a83b service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:22.149308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e0a3e-b8d0-46d1-9823-280fb2a0a83b service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:22.149588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e0a3e-b8d0-46d1-9823-280fb2a0a83b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:22.151252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-996e0a3e-b8d0-46d1-9823-280fb2a0a83b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:22.241277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e0a3e-b8d0-46d1-9823-280fb2a0a83b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:22.242955 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e0a3e-b8d0-46d1-9823-280fb2a0a83b service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:22.261529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 740/1481] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:08:24.465452 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd4fb7-6e9e-4a33-859b-208f77b98b00 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:24.465998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd4fb7-6e9e-4a33-859b-208f77b98b00 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:24.465998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd4fb7-6e9e-4a33-859b-208f77b98b00 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:24.467392 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 742/1482] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:08:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:08:24.472817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cffa13-8cb1-40c9-96b0-526668ef7dfe None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:24.473880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cffa13-8cb1-40c9-96b0-526668ef7dfe None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:24.473880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cffa13-8cb1-40c9-96b0-526668ef7dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:24.539610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d4cffa13-8cb1-40c9-96b0-526668ef7dfe None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:08:24.540836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d4cffa13-8cb1-40c9-96b0-526668ef7dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:24.589161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:24.639848 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 741/1483] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:08:24.677195 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd4fb7-6e9e-4a33-859b-208f77b98b00 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:24.678740 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a4bd4fb7-6e9e-4a33-859b-208f77b98b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:24.701668 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:24.781951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd4fb7-6e9e-4a33-859b-208f77b98b00 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:24.856868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84d962-cbf0-4ab4-ac77-cd1fe713c009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:24.857627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84d962-cbf0-4ab4-ac77-cd1fe713c009 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:24.857972 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84d962-cbf0-4ab4-ac77-cd1fe713c009 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:24.858251 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84d962-cbf0-4ab4-ac77-cd1fe713c009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:24.860047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2b84d962-cbf0-4ab4-ac77-cd1fe713c009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:24.936397 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:24.971958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84d962-cbf0-4ab4-ac77-cd1fe713c009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:24.973517 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b84d962-cbf0-4ab4-ac77-cd1fe713c009 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:24.992292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 743/1484] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:08:27.076326 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a69b4c-08ae-4c76-953e-c4e817630690 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:27.076927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a69b4c-08ae-4c76-953e-c4e817630690 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:27.076927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a69b4c-08ae-4c76-953e-c4e817630690 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:27.078251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 742/1485] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:08:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:08:27.084136 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de554af-bd67-461b-96f9-80408a88b950 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:27.084567 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de554af-bd67-461b-96f9-80408a88b950 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:27.084695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de554af-bd67-461b-96f9-80408a88b950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:27.153317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-6de554af-bd67-461b-96f9-80408a88b950 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:08:27.154949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6de554af-bd67-461b-96f9-80408a88b950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:27.254259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 744/1486] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:08:27.292889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a69b4c-08ae-4c76-953e-c4e817630690 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:27.294423 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c4a69b4c-08ae-4c76-953e-c4e817630690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:27.399665 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a69b4c-08ae-4c76-953e-c4e817630690 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:27.471788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:27.473779 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a13f46-4e49-4525-a7c8-6c89db587bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:27.474396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a13f46-4e49-4525-a7c8-6c89db587bcf service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:27.474889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a13f46-4e49-4525-a7c8-6c89db587bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:27.475331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a13f46-4e49-4525-a7c8-6c89db587bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:27.477153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c7a13f46-4e49-4525-a7c8-6c89db587bcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:27.587313 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a13f46-4e49-4525-a7c8-6c89db587bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:27.588739 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a13f46-4e49-4525-a7c8-6c89db587bcf service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:27.606547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 743/1487] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:08:29.697687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead40f3b-a86a-49de-b05e-7ab9dc1f98b6 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:29.698254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead40f3b-a86a-49de-b05e-7ab9dc1f98b6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:29.698254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead40f3b-a86a-49de-b05e-7ab9dc1f98b6 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:29.699725 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 745/1488] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:08:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:08:29.705088 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cc572-2630-472b-9d28-2f0e16dcaca3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:29.705562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cc572-2630-472b-9d28-2f0e16dcaca3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:29.705669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cc572-2630-472b-9d28-2f0e16dcaca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:29.772773 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3a1cc572-2630-472b-9d28-2f0e16dcaca3 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:08:29.773999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3a1cc572-2630-472b-9d28-2f0e16dcaca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:29.826544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:29.875406 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 744/1489] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:08:29.915469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead40f3b-a86a-49de-b05e-7ab9dc1f98b6 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:29.917003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ead40f3b-a86a-49de-b05e-7ab9dc1f98b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:29.952736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:30.021634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead40f3b-a86a-49de-b05e-7ab9dc1f98b6 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:30.100158 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4dcc6b-15aa-4e95-b2d4-ca97962ec752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:30.100776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dcc6b-15aa-4e95-b2d4-ca97962ec752 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:30.101058 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dcc6b-15aa-4e95-b2d4-ca97962ec752 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:30.101329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4dcc6b-15aa-4e95-b2d4-ca97962ec752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:30.102982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-da4dcc6b-15aa-4e95-b2d4-ca97962ec752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:30.184702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:30.214236 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dcc6b-15aa-4e95-b2d4-ca97962ec752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:30.215722 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4dcc6b-15aa-4e95-b2d4-ca97962ec752 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:30.235191 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 746/1490] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:08:32.472747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe99687-7ce2-4ff8-8971-a99b9ccb55f8 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:32.473703 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe99687-7ce2-4ff8-8971-a99b9ccb55f8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:32.473703 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe99687-7ce2-4ff8-8971-a99b9ccb55f8 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:32.475869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 745/1491] 10.222.0.100 () {54 vars in 931 bytes} [Mon Aug 1 15:08:32 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:08:32.480969 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d2555-01f9-4114-8bac-bcc1d60a77e0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:32.481401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d2555-01f9-4114-8bac-bcc1d60a77e0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:32.481518 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d2555-01f9-4114-8bac-bcc1d60a77e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:32.549049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-868d2555-01f9-4114-8bac-bcc1d60a77e0 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:08:32.550431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-868d2555-01f9-4114-8bac-bcc1d60a77e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:32.650624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 747/1492] 10.222.0.100 () {58 vars in 1028 bytes} [Mon Aug 1 15:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:08:32.690177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe99687-7ce2-4ff8-8971-a99b9ccb55f8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:08:32.692304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dfe99687-7ce2-4ff8-8971-a99b9ccb55f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:32.798412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe99687-7ce2-4ff8-8971-a99b9ccb55f8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:08:32.871825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f28cb-b505-4eee-8e90-b024945d2c6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:08:32.872445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f28cb-b505-4eee-8e90-b024945d2c6a service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:08:32.872750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f28cb-b505-4eee-8e90-b024945d2c6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:08:32.873025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f28cb-b505-4eee-8e90-b024945d2c6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:08:32.874917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d05f28cb-b505-4eee-8e90-b024945d2c6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:08:32.885093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:32.991764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f28cb-b505-4eee-8e90-b024945d2c6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:08:32.993216 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f28cb-b505-4eee-8e90-b024945d2c6a service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:08:33.011580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 746/1493] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:08:42.648821 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=82 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:08:43.008754 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=50 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:34.236944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109d106-5603-400f-83f2-4b5023d0f7d0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:34.237550 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109d106-5603-400f-83f2-4b5023d0f7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:34.237550 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109d106-5603-400f-83f2-4b5023d0f7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:34.316931 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3109d106-5603-400f-83f2-4b5023d0f7d0 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:34.318141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3109d106-5603-400f-83f2-4b5023d0f7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:34.422136 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 748/1494] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:10:34.430924 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095f12-db0b-48e8-a645-c2fc8e89711c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:34.431443 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095f12-db0b-48e8-a645-c2fc8e89711c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:34.431552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095f12-db0b-48e8-a645-c2fc8e89711c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:34.452493 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-74095f12-db0b-48e8-a645-c2fc8e89711c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:34.453701 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-74095f12-db0b-48e8-a645-c2fc8e89711c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:34.463046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 747/1495] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:10:34.467631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109d106-5603-400f-83f2-4b5023d0f7d0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:34.478063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109d106-5603-400f-83f2-4b5023d0f7d0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:34.482644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2f968-15f4-4ac1-bd51-7f4e4f35b9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:34.483285 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2f968-15f4-4ac1-bd51-7f4e4f35b9d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:34.483568 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2f968-15f4-4ac1-bd51-7f4e4f35b9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:34.483846 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2f968-15f4-4ac1-bd51-7f4e4f35b9d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:34.484544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2f968-15f4-4ac1-bd51-7f4e4f35b9d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:10:34.484851 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2f968-15f4-4ac1-bd51-7f4e4f35b9d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:34.485877 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-b1d2f968-15f4-4ac1-bd51-7f4e4f35b9d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:10:34.485877 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Aug 01 15:10:34.485877 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: Aug 01 15:10:34.487357 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2f968-15f4-4ac1-bd51-7f4e4f35b9d2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:34.499301 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 749/1496] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:10:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:10:34.509160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74095f12-db0b-48e8-a645-c2fc8e89711c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:34.519045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74095f12-db0b-48e8-a645-c2fc8e89711c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:34.524025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adf09c-dac3-4e71-b52e-4454469136f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:34.524633 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adf09c-dac3-4e71-b52e-4454469136f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:34.524917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adf09c-dac3-4e71-b52e-4454469136f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:34.525190 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adf09c-dac3-4e71-b52e-4454469136f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:34.525872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adf09c-dac3-4e71-b52e-4454469136f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:34.527634 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-67adf09c-dac3-4e71-b52e-4454469136f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:10:34.527634 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Aug 01 15:10:34.527634 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: Aug 01 15:10:34.528828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adf09c-dac3-4e71-b52e-4454469136f7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:34.567200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 748/1497] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:10:34 2022] POST /identity/v3/projects => generated 380 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:10:34.571412 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2f968-15f4-4ac1-bd51-7f4e4f35b9d2 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:34.581369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2f968-15f4-4ac1-bd51-7f4e4f35b9d2 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:34.631764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:34.639344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0c034-85a1-4f9f-9e58-cae4b08dbbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:34.639960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0c034-85a1-4f9f-9e58-cae4b08dbbb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:34.640269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0c034-85a1-4f9f-9e58-cae4b08dbbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:34.640545 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0c034-85a1-4f9f-9e58-cae4b08dbbb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:34.641217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0c034-85a1-4f9f-9e58-cae4b08dbbb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:34.642224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-66d0c034-85a1-4f9f-9e58-cae4b08dbbb2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:10:34.642224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Aug 01 15:10:34.642224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: Aug 01 15:10:34.643480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0c034-85a1-4f9f-9e58-cae4b08dbbb2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:34.663752 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 750/1498] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:10:34 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:10:34.668081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adf09c-dac3-4e71-b52e-4454469136f7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:34.680547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adf09c-dac3-4e71-b52e-4454469136f7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:34.685773 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0ae6f-0d6f-43db-8fae-edf6891f6106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:34.686505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0ae6f-0d6f-43db-8fae-edf6891f6106 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:34.686932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0ae6f-0d6f-43db-8fae-edf6891f6106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:34.687213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0ae6f-0d6f-43db-8fae-edf6891f6106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:34.687991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0ae6f-0d6f-43db-8fae-edf6891f6106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:34.689115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-9aa0ae6f-0d6f-43db-8fae-edf6891f6106 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:10:34.689115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Aug 01 15:10:34.689115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: Aug 01 15:10:34.690624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0ae6f-0d6f-43db-8fae-edf6891f6106 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:34.706330 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 749/1499] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:10:34 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:10:34.710761 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0c034-85a1-4f9f-9e58-cae4b08dbbb2 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:34.721077 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0c034-85a1-4f9f-9e58-cae4b08dbbb2 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:34.726022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5aa87c-b34e-42eb-90d2-c90ed3189417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:34.726644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5aa87c-b34e-42eb-90d2-c90ed3189417 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:34.727050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5aa87c-b34e-42eb-90d2-c90ed3189417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:34.727329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5aa87c-b34e-42eb-90d2-c90ed3189417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:34.728028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5aa87c-b34e-42eb-90d2-c90ed3189417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:34.729049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-dc5aa87c-b34e-42eb-90d2-c90ed3189417 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:10:34.729049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Aug 01 15:10:34.729049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: Aug 01 15:10:34.729945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5aa87c-b34e-42eb-90d2-c90ed3189417 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:34.745640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 751/1500] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:10:34 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:10:34.749814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0ae6f-0d6f-43db-8fae-edf6891f6106 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:34.760753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0ae6f-0d6f-43db-8fae-edf6891f6106 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:34.765441 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc5170-ca1a-494a-a84c-36bea06cf0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:34.766120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc5170-ca1a-494a-a84c-36bea06cf0cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:34.766405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc5170-ca1a-494a-a84c-36bea06cf0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:34.766709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc5170-ca1a-494a-a84c-36bea06cf0cc admin admin] PATH_INFO: `/v3/projects/1c02b9a2f8494ea2a929eb219fad7c0b/users/608ea6b79fc74fc782999cf00324b9ad/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:34.781955 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc5170-ca1a-494a-a84c-36bea06cf0cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c02b9a2f8494ea2a929eb219fad7c0b, user_id=608ea6b79fc74fc782999cf00324b9ad, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:34.783034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-12fc5170-ca1a-494a-a84c-36bea06cf0cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:10:34.783034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Aug 01 15:10:34.783034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: Aug 01 15:10:34.785025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc5170-ca1a-494a-a84c-36bea06cf0cc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:34.799496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 750/1501] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:10:34 2022] PUT /identity/v3/projects/1c02b9a2f8494ea2a929eb219fad7c0b/users/608ea6b79fc74fc782999cf00324b9ad/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:34.811635 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a224c-d671-4d91-8816-435ab9dd6186 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:34.811922 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a224c-d671-4d91-8816-435ab9dd6186 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:34.812200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a224c-d671-4d91-8816-435ab9dd6186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:34.841294 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ef5a224c-d671-4d91-8816-435ab9dd6186 None None] MFA Rules not processed for user `608ea6b79fc74fc782999cf00324b9ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:34.842503 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ef5a224c-d671-4d91-8816-435ab9dd6186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:34.905669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 752/1502] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:10:34.919272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7355a-3dc1-4b3f-9f49-05e22151e7ea None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:34.919570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7355a-3dc1-4b3f-9f49-05e22151e7ea None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:34.919854 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7355a-3dc1-4b3f-9f49-05e22151e7ea None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:34.921201 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 751/1503] 10.222.0.100 () {54 vars in 969 bytes} [Mon Aug 1 15:10:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:10:34.926566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4e734-56a1-44ce-835e-1a6af0de8f63 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:34.926884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4e734-56a1-44ce-835e-1a6af0de8f63 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:34.927166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4e734-56a1-44ce-835e-1a6af0de8f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:34.988597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f9c4e734-56a1-44ce-835e-1a6af0de8f63 None None] MFA Rules not processed for user `ead1b36d35f3485f89edd15b8443feea`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:34.989825 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f9c4e734-56a1-44ce-835e-1a6af0de8f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:35.065776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 753/1504] 10.222.0.100 () {58 vars in 1066 bytes} [Mon Aug 1 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:10:35.073494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7355a-3dc1-4b3f-9f49-05e22151e7ea None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:35.084757 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7355a-3dc1-4b3f-9f49-05e22151e7ea None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:35.090070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845b22b-5b16-4f3e-b3c0-eb7aea791cb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:35.091490 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845b22b-5b16-4f3e-b3c0-eb7aea791cb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:35.091769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845b22b-5b16-4f3e-b3c0-eb7aea791cb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:35.092043 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845b22b-5b16-4f3e-b3c0-eb7aea791cb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:35.100739 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845b22b-5b16-4f3e-b3c0-eb7aea791cb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:35.102157 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f845b22b-5b16-4f3e-b3c0-eb7aea791cb5 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:35.112317 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 752/1505] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:10:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:10:35.174268 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1ebd8-9df1-4217-8527-b2a326973f5f None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:35.174610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1ebd8-9df1-4217-8527-b2a326973f5f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:35.174993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1ebd8-9df1-4217-8527-b2a326973f5f None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:35.176626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 754/1506] 10.222.0.100 () {52 vars in 835 bytes} [Mon Aug 1 15:10:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:10:35.181274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845b22b-5b16-4f3e-b3c0-eb7aea791cb5 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:35.278185 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845b22b-5b16-4f3e-b3c0-eb7aea791cb5 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:35.283431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:35.284217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:35.284600 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:35.285040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] PATH_INFO: `/v3/users/608ea6b79fc74fc782999cf00324b9ad` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:35.286386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=608ea6b79fc74fc782999cf00324b9ad)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:35.287538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:10:35.287538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Aug 01 15:10:35.287538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: Aug 01 15:10:35.288929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:35.377808 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '608ea6b79fc74fc782999cf00324b9ad'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:10:35.379006 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '608ea6b79fc74fc782999cf00324b9ad'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:10:35.402543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '608ea6b79fc74fc782999cf00324b9ad'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:10:35.403707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '608ea6b79fc74fc782999cf00324b9ad'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:10:35.441851 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '608ea6b79fc74fc782999cf00324b9ad'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:10:35.448799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:35.466069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 753/1507] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:10:35 2022] DELETE /identity/v3/users/608ea6b79fc74fc782999cf00324b9ad => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:35.471102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4d774-398a-40e9-a434-4d062ecb00d4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:35.471777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4d774-398a-40e9-a434-4d062ecb00d4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:35.471777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4d774-398a-40e9-a434-4d062ecb00d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:35.498844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-27a4d774-398a-40e9-a434-4d062ecb00d4 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:10:35.500796 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-27a4d774-398a-40e9-a434-4d062ecb00d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:35.517065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:35.601090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 755/1508] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:10:35.612830 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:35.615044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:35.719920 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e18e34-2f47-4066-8864-5844667860f4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:35.724328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca94f6e-0fba-4265-aff1-4b1178d867ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:35.724997 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca94f6e-0fba-4265-aff1-4b1178d867ec service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:35.725280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca94f6e-0fba-4265-aff1-4b1178d867ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:35.725560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca94f6e-0fba-4265-aff1-4b1178d867ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:35.735162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca94f6e-0fba-4265-aff1-4b1178d867ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:35.737210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca94f6e-0fba-4265-aff1-4b1178d867ec service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:35.747056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 754/1509] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:10:36.152984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4d774-398a-40e9-a434-4d062ecb00d4 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:10:36.154649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-27a4d774-398a-40e9-a434-4d062ecb00d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:10:36.167824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4d774-398a-40e9-a434-4d062ecb00d4 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:10:36.176705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305cce8-02d0-42c8-86e7-42697c0dcf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:10:36.177426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305cce8-02d0-42c8-86e7-42697c0dcf4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:10:36.177716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305cce8-02d0-42c8-86e7-42697c0dcf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:10:36.178000 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305cce8-02d0-42c8-86e7-42697c0dcf4a admin admin] PATH_INFO: `/v3/projects/1c02b9a2f8494ea2a929eb219fad7c0b` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:10:36.179281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305cce8-02d0-42c8-86e7-42697c0dcf4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c02b9a2f8494ea2a929eb219fad7c0b)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:10:36.180332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-0305cce8-02d0-42c8-86e7-42697c0dcf4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:10:36.180332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Aug 01 15:10:36.180332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: Aug 01 15:10:36.181557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305cce8-02d0-42c8-86e7-42697c0dcf4a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:10:36.258471 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0305cce8-02d0-42c8-86e7-42697c0dcf4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c02b9a2f8494ea2a929eb219fad7c0b', 'request_id': 'req-0305cce8-02d0-42c8-86e7-42697c0dcf4a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:10:36.271812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0305cce8-02d0-42c8-86e7-42697c0dcf4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c02b9a2f8494ea2a929eb219fad7c0b', 'request_id': 'req-0305cce8-02d0-42c8-86e7-42697c0dcf4a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:10:36.282136 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0305cce8-02d0-42c8-86e7-42697c0dcf4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c02b9a2f8494ea2a929eb219fad7c0b', 'request_id': 'req-0305cce8-02d0-42c8-86e7-42697c0dcf4a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:10:36.293925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0305cce8-02d0-42c8-86e7-42697c0dcf4a admin admin] The token cache is being invalidate because project 1c02b9a2f8494ea2a929eb219fad7c0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:10:36.294220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0305cce8-02d0-42c8-86e7-42697c0dcf4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:10:36.294891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0305cce8-02d0-42c8-86e7-42697c0dcf4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:10:36.296535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 756/1510] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:10:36 2022] DELETE /identity/v3/projects/1c02b9a2f8494ea2a929eb219fad7c0b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:10:45.742550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:10:46.290926 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:21.046689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00850bb8-58a4-4023-b71b-9b1b3944d3b1 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:21.047305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00850bb8-58a4-4023-b71b-9b1b3944d3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:21.047305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00850bb8-58a4-4023-b71b-9b1b3944d3b1 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:21.048651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:12:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:12:21.054703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d6c99-c728-4886-93fd-2a05f7723c6f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:21.055238 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d6c99-c728-4886-93fd-2a05f7723c6f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:21.055238 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d6c99-c728-4886-93fd-2a05f7723c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:21.074251 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c98d6c99-c728-4886-93fd-2a05f7723c6f None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:21.075495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c98d6c99-c728-4886-93fd-2a05f7723c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:21.137220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:21.294170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00850bb8-58a4-4023-b71b-9b1b3944d3b1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:21.295935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-00850bb8-58a4-4023-b71b-9b1b3944d3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:21.407526 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00850bb8-58a4-4023-b71b-9b1b3944d3b1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:21.414171 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b01605-9294-49b4-91dc-a7bcff71ddf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:21.414871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b01605-9294-49b4-91dc-a7bcff71ddf7 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:21.415155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b01605-9294-49b4-91dc-a7bcff71ddf7 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:21.415430 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b01605-9294-49b4-91dc-a7bcff71ddf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:21.424509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b01605-9294-49b4-91dc-a7bcff71ddf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:21.426615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b01605-9294-49b4-91dc-a7bcff71ddf7 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:21.439184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 756/1513] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:12:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:22.281303 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258bad0-b138-4e7c-a5a7-39040085748f None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:22.281911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258bad0-b138-4e7c-a5a7-39040085748f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:22.282263 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b258bad0-b138-4e7c-a5a7-39040085748f None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:22.284755 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:12:22 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:12:22.290076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef566e-7092-4d01-bbe6-75e0d1b6ef7b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:22.290559 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef566e-7092-4d01-bbe6-75e0d1b6ef7b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:22.290691 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef566e-7092-4d01-bbe6-75e0d1b6ef7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:22.314561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c6ef566e-7092-4d01-bbe6-75e0d1b6ef7b None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:22.315838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c6ef566e-7092-4d01-bbe6-75e0d1b6ef7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:22.325753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:22.481176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258bad0-b138-4e7c-a5a7-39040085748f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:22.495302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b258bad0-b138-4e7c-a5a7-39040085748f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:22.500691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d384a1b-15c9-42d0-b910-b4b56ba47a6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:22.501314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d384a1b-15c9-42d0-b910-b4b56ba47a6a service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:22.501601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d384a1b-15c9-42d0-b910-b4b56ba47a6a service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:22.501908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d384a1b-15c9-42d0-b910-b4b56ba47a6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:22.511690 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d384a1b-15c9-42d0-b910-b4b56ba47a6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:22.513322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d384a1b-15c9-42d0-b910-b4b56ba47a6a service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:22.524370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 759/1516] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:12:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:23.338062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b96fa-bdae-4a65-9114-aa09a58e90fd None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:23.338614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b96fa-bdae-4a65-9114-aa09a58e90fd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:23.338614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b96fa-bdae-4a65-9114-aa09a58e90fd None None] PATH_INFO: `/v3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:23.339933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 758/1517] 10.222.0.97 () {54 vars in 929 bytes} [Mon Aug 1 15:12:23 2022] GET /identity/v3 => generated 256 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:12:23.347892 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd231ae-e183-4eac-8a13-b5648efa2824 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:23.348342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd231ae-e183-4eac-8a13-b5648efa2824 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:23.348488 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd231ae-e183-4eac-8a13-b5648efa2824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:23.398258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7cd231ae-e183-4eac-8a13-b5648efa2824 None None] MFA Rules not processed for user `3bac5863e7bc4c7e854bd60912adb561`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:23.399544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7cd231ae-e183-4eac-8a13-b5648efa2824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:23.473442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 760/1518] 10.222.0.97 () {58 vars in 1019 bytes} [Mon Aug 1 15:12:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:32.306118 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=49 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:32.869602 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a35da9-b788-46f6-8487-3a9b3a48c9ed None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.869816 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a35da9-b788-46f6-8487-3a9b3a48c9ed None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.870068 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a35da9-b788-46f6-8487-3a9b3a48c9ed None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.871447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 759/1519] 10.222.0.100 () {54 vars in 966 bytes} [Mon Aug 1 15:12:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:12:32.876520 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b4522-d414-4f53-956f-9c822252069a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.877040 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b4522-d414-4f53-956f-9c822252069a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.877149 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b4522-d414-4f53-956f-9c822252069a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.896612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-216b4522-d414-4f53-956f-9c822252069a None None] MFA Rules not processed for user `3bac5863e7bc4c7e854bd60912adb561`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:32.897856 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-216b4522-d414-4f53-956f-9c822252069a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:32.906629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 761/1520] 10.222.0.100 () {58 vars in 1063 bytes} [Mon Aug 1 15:12:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:32.915129 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a35da9-b788-46f6-8487-3a9b3a48c9ed None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:32.929538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a35da9-b788-46f6-8487-3a9b3a48c9ed None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:32.934541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4115779-aa61-49e8-845b-f33e18a9622d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:32.935212 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4115779-aa61-49e8-845b-f33e18a9622d service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:32.935499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4115779-aa61-49e8-845b-f33e18a9622d service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:32.935777 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4115779-aa61-49e8-845b-f33e18a9622d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:32.945317 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4115779-aa61-49e8-845b-f33e18a9622d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:32.946952 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4115779-aa61-49e8-845b-f33e18a9622d service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:32.956987 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 760/1521] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:12:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:33.804355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d9468-2422-44b9-97eb-23be71d987a9 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.804685 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d9468-2422-44b9-97eb-23be71d987a9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.805055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5d9468-2422-44b9-97eb-23be71d987a9 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.806436 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:12:33 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:12:33.811300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d24fdd-4654-415c-a97e-b334c2d2174c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:33.811844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d24fdd-4654-415c-a97e-b334c2d2174c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:33.811844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d24fdd-4654-415c-a97e-b334c2d2174c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:33.831057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-34d24fdd-4654-415c-a97e-b334c2d2174c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:33.832329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-34d24fdd-4654-415c-a97e-b334c2d2174c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:33.840530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:33.999642 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d9468-2422-44b9-97eb-23be71d987a9 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:34.017535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5d9468-2422-44b9-97eb-23be71d987a9 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:34.027324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d0d56-3a53-4b27-beaf-1c7b5455922d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:34.028346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d0d56-3a53-4b27-beaf-1c7b5455922d service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.028823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d0d56-3a53-4b27-beaf-1c7b5455922d service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.029295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d0d56-3a53-4b27-beaf-1c7b5455922d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.042582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d0d56-3a53-4b27-beaf-1c7b5455922d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:34.044240 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d0d56-3a53-4b27-beaf-1c7b5455922d service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:34.055002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 763/1524] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:34.915225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bdfc6-c3fe-4230-b808-3f5650617a46 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.915888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bdfc6-c3fe-4230-b808-3f5650617a46 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.915888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bdfc6-c3fe-4230-b808-3f5650617a46 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.917225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 15:12:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:12:34.925601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe66560-6f73-4781-b326-1d2b6ba314fd None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:34.926723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe66560-6f73-4781-b326-1d2b6ba314fd None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:34.927267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe66560-6f73-4781-b326-1d2b6ba314fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:34.958973 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4fe66560-6f73-4781-b326-1d2b6ba314fd None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:34.960226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4fe66560-6f73-4781-b326-1d2b6ba314fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:34.969126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 15:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:35.142700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bdfc6-c3fe-4230-b808-3f5650617a46 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:35.158897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bdfc6-c3fe-4230-b808-3f5650617a46 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:35.166514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac258795-2cf3-4fde-a2a9-6c1836ef4cba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:35.167100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac258795-2cf3-4fde-a2a9-6c1836ef4cba service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:35.167384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac258795-2cf3-4fde-a2a9-6c1836ef4cba service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:35.167657 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac258795-2cf3-4fde-a2a9-6c1836ef4cba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:35.177535 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac258795-2cf3-4fde-a2a9-6c1836ef4cba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:35.179218 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac258795-2cf3-4fde-a2a9-6c1836ef4cba service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:35.189822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 763/1527] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:12:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:39.423449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b1ea0-65bb-43f3-83eb-55ec8cb5bc03 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:39.424091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b1ea0-65bb-43f3-83eb-55ec8cb5bc03 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:39.424091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b1ea0-65bb-43f3-83eb-55ec8cb5bc03 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:39.425378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:12:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:12:39.431276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda488b-5d5f-4b87-8558-02e46ae94826 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:39.431715 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda488b-5d5f-4b87-8558-02e46ae94826 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:39.431865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda488b-5d5f-4b87-8558-02e46ae94826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:39.453304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2bda488b-5d5f-4b87-8558-02e46ae94826 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:39.454535 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2bda488b-5d5f-4b87-8558-02e46ae94826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:39.464123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:39.544132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b1ea0-65bb-43f3-83eb-55ec8cb5bc03 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:39.545865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e64b1ea0-65bb-43f3-83eb-55ec8cb5bc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:39.575589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:39.609094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b1ea0-65bb-43f3-83eb-55ec8cb5bc03 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:39.616049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a235fe-f6bb-44de-ade4-100bac0b4088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:39.617021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a235fe-f6bb-44de-ade4-100bac0b4088 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:39.617432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a235fe-f6bb-44de-ade4-100bac0b4088 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:39.617756 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a235fe-f6bb-44de-ade4-100bac0b4088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:39.631658 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a235fe-f6bb-44de-ade4-100bac0b4088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:39.633470 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a235fe-f6bb-44de-ade4-100bac0b4088 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:39.645196 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 766/1530] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:41.720544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d1d82-fd32-40b2-8a1c-bc09f517cc9c None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:41.721341 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d1d82-fd32-40b2-8a1c-bc09f517cc9c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:41.721451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d1d82-fd32-40b2-8a1c-bc09f517cc9c None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:41.723572 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:12:41 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:12:41.730699 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536f7f0-c94b-492a-a9f7-070c8343e49a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:41.731249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536f7f0-c94b-492a-a9f7-070c8343e49a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:41.731249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536f7f0-c94b-492a-a9f7-070c8343e49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:41.751567 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3536f7f0-c94b-492a-a9f7-070c8343e49a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:41.752784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3536f7f0-c94b-492a-a9f7-070c8343e49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:41.761979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:41.801601 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d1d82-fd32-40b2-8a1c-bc09f517cc9c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:41.803373 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-276d1d82-fd32-40b2-8a1c-bc09f517cc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:41.817116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d1d82-fd32-40b2-8a1c-bc09f517cc9c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:41.821099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ccf72-e1fe-4bee-95bd-5ffea824e74a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:41.821744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ccf72-e1fe-4bee-95bd-5ffea824e74a service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:41.822061 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ccf72-e1fe-4bee-95bd-5ffea824e74a service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:41.822341 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ccf72-e1fe-4bee-95bd-5ffea824e74a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:41.832334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ccf72-e1fe-4bee-95bd-5ffea824e74a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:41.834055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ccf72-e1fe-4bee-95bd-5ffea824e74a service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:41.844331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 766/1533] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:12:45.511486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1840631-4e5d-4dd7-bb72-a867faf0fa26 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:45.512216 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1840631-4e5d-4dd7-bb72-a867faf0fa26 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:45.512216 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1840631-4e5d-4dd7-bb72-a867faf0fa26 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:45.513409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:12:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:12:45.518721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3123d-9877-4534-b616-9d9086c58d27 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:45.519200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3123d-9877-4534-b616-9d9086c58d27 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:45.519862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3123d-9877-4534-b616-9d9086c58d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:45.540770 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b4f3123d-9877-4534-b616-9d9086c58d27 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:45.541998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b4f3123d-9877-4534-b616-9d9086c58d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:45.550039 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:45.864768 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1840631-4e5d-4dd7-bb72-a867faf0fa26 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:45.866874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c1840631-4e5d-4dd7-bb72-a867faf0fa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:45.933505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1840631-4e5d-4dd7-bb72-a867faf0fa26 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:45.937740 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194db35d-2aac-48d9-a247-5843aff50afe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:45.938693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db35d-2aac-48d9-a247-5843aff50afe service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:45.938693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db35d-2aac-48d9-a247-5843aff50afe service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:45.938989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db35d-2aac-48d9-a247-5843aff50afe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:45.949098 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194db35d-2aac-48d9-a247-5843aff50afe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:45.950559 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194db35d-2aac-48d9-a247-5843aff50afe service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:45.961512 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 769/1536] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:48.063033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50534444-3ffb-4ff3-98f0-2955803e1243 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:48.063575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50534444-3ffb-4ff3-98f0-2955803e1243 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:48.063575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50534444-3ffb-4ff3-98f0-2955803e1243 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:48.064877 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 15:12:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:12:48.071017 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cc5e8-2763-476f-82c6-e65b14b5f2cb None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:48.071541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cc5e8-2763-476f-82c6-e65b14b5f2cb None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:48.071541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cc5e8-2763-476f-82c6-e65b14b5f2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:48.092979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-6c5cc5e8-2763-476f-82c6-e65b14b5f2cb None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:12:48.094030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6c5cc5e8-2763-476f-82c6-e65b14b5f2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:48.102228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 15:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:12:48.420088 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50534444-3ffb-4ff3-98f0-2955803e1243 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:12:48.422019 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-50534444-3ffb-4ff3-98f0-2955803e1243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:12:48.436459 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50534444-3ffb-4ff3-98f0-2955803e1243 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:12:48.440131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca180f4-4e52-4bda-95b3-21d6552b51ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:12:48.440749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca180f4-4e52-4bda-95b3-21d6552b51ca service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:12:48.441088 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca180f4-4e52-4bda-95b3-21d6552b51ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:12:48.441911 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca180f4-4e52-4bda-95b3-21d6552b51ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:12:48.451594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca180f4-4e52-4bda-95b3-21d6552b51ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:12:48.453045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca180f4-4e52-4bda-95b3-21d6552b51ca service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:12:48.463449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 769/1539] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:12:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:12:58.087131 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=47 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:12:58.458993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=33 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:22.064138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d8207-b8ec-4963-9ac6-e75a3099ff05 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.065640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d8207-b8ec-4963-9ac6-e75a3099ff05 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.066166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d8207-b8ec-4963-9ac6-e75a3099ff05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.091335 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-673d8207-b8ec-4963-9ac6-e75a3099ff05 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.093396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-673d8207-b8ec-4963-9ac6-e75a3099ff05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.107104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 771/1540] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.115906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197b3e1-381a-4bb5-a63b-7fa96e52c921 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.116515 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197b3e1-381a-4bb5-a63b-7fa96e52c921 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.116515 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197b3e1-381a-4bb5-a63b-7fa96e52c921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.150974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7197b3e1-381a-4bb5-a63b-7fa96e52c921 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.153010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7197b3e1-381a-4bb5-a63b-7fa96e52c921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.171028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 770/1541] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.176288 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d8207-b8ec-4963-9ac6-e75a3099ff05 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:22.188245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d8207-b8ec-4963-9ac6-e75a3099ff05 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:22.197343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1b920-8c0d-4aea-892a-badf38f58ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:22.197979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1b920-8c0d-4aea-892a-badf38f58ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.198267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1b920-8c0d-4aea-892a-badf38f58ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.198546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1b920-8c0d-4aea-892a-badf38f58ce2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.198684 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749bd52-7635-44d6-96a1-81058d571c25 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.198943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749bd52-7635-44d6-96a1-81058d571c25 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.199194 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749bd52-7635-44d6-96a1-81058d571c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.199453 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1b920-8c0d-4aea-892a-badf38f58ce2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:22.199770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1b920-8c0d-4aea-892a-badf38f58ce2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:22.201391 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1b920-8c0d-4aea-892a-badf38f58ce2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:22.224078 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 772/1542] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:22.233447 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1b920-8c0d-4aea-892a-badf38f58ce2 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:22.244285 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1b920-8c0d-4aea-892a-badf38f58ce2 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:22.253868 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b749bd52-7635-44d6-96a1-81058d571c25 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.255138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b749bd52-7635-44d6-96a1-81058d571c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.261904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91426cdc-b51c-431e-a522-6c51f797b95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:22.262553 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91426cdc-b51c-431e-a522-6c51f797b95b admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.262870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91426cdc-b51c-431e-a522-6c51f797b95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.263146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91426cdc-b51c-431e-a522-6c51f797b95b admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.264029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91426cdc-b51c-431e-a522-6c51f797b95b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:22.265066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-91426cdc-b51c-431e-a522-6c51f797b95b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:14:22.265066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Aug 01 15:14:22.265066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: Aug 01 15:14:22.266278 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91426cdc-b51c-431e-a522-6c51f797b95b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:22.276657 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 771/1543] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:22.278543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da5066-ea02-40cb-bb22-8079aa208a3f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.278863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da5066-ea02-40cb-bb22-8079aa208a3f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.279140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da5066-ea02-40cb-bb22-8079aa208a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.324375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 773/1544] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/projects => generated 376 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:14:22.324702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c1da5066-ea02-40cb-bb22-8079aa208a3f None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.325758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9ca40-6f10-45dc-a5c7-aaa22c3efc81 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.325956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c1da5066-ea02-40cb-bb22-8079aa208a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.326070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9ca40-6f10-45dc-a5c7-aaa22c3efc81 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.326340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d9ca40-6f10-45dc-a5c7-aaa22c3efc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.369734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-24d9ca40-6f10-45dc-a5c7-aaa22c3efc81 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.371555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-24d9ca40-6f10-45dc-a5c7-aaa22c3efc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.442494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 772/1545] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.443669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da5066-ea02-40cb-bb22-8079aa208a3f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:22.459956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da5066-ea02-40cb-bb22-8079aa208a3f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:22.475745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dae7a1-61a9-45de-aaff-2ee39fdca82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:22.476397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dae7a1-61a9-45de-aaff-2ee39fdca82b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.476687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dae7a1-61a9-45de-aaff-2ee39fdca82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.476966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dae7a1-61a9-45de-aaff-2ee39fdca82b admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.477654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dae7a1-61a9-45de-aaff-2ee39fdca82b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:22.478710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-90dae7a1-61a9-45de-aaff-2ee39fdca82b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:14:22.478710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Aug 01 15:14:22.478710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: Aug 01 15:14:22.479931 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dae7a1-61a9-45de-aaff-2ee39fdca82b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:22.498568 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 774/1546] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.500048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dab117-c9b7-41a4-86ad-b9e56d1ee430 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.500334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dab117-c9b7-41a4-86ad-b9e56d1ee430 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.500770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dab117-c9b7-41a4-86ad-b9e56d1ee430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.526020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 773/1547] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/users => generated 289 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:22.527451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a198f7-328e-46ef-ade0-de5d794b8fbe None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.527743 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a198f7-328e-46ef-ade0-de5d794b8fbe None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.528021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a198f7-328e-46ef-ade0-de5d794b8fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.536849 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-55dab117-c9b7-41a4-86ad-b9e56d1ee430 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.538392 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-55dab117-c9b7-41a4-86ad-b9e56d1ee430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.551274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 775/1548] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.551569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dab117-c9b7-41a4-86ad-b9e56d1ee430 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:22.552707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-88a198f7-328e-46ef-ade0-de5d794b8fbe None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.554023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-88a198f7-328e-46ef-ade0-de5d794b8fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.563817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dab117-c9b7-41a4-86ad-b9e56d1ee430 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:22.568383 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 774/1549] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.569271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee7466-606b-48cb-a7f7-bf40c880777d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:22.569651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c1d02-46cf-4d49-a18b-b97592ac93d2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.569894 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee7466-606b-48cb-a7f7-bf40c880777d admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.570025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c1d02-46cf-4d49-a18b-b97592ac93d2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.570195 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee7466-606b-48cb-a7f7-bf40c880777d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.570308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c1d02-46cf-4d49-a18b-b97592ac93d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.570469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee7466-606b-48cb-a7f7-bf40c880777d admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.571170 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee7466-606b-48cb-a7f7-bf40c880777d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:22.571471 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee7466-606b-48cb-a7f7-bf40c880777d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:22.573286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee7466-606b-48cb-a7f7-bf40c880777d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:22.586393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 776/1550] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:22.587499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee7466-606b-48cb-a7f7-bf40c880777d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:22.589769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-055c1d02-46cf-4d49-a18b-b97592ac93d2 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.591011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-055c1d02-46cf-4d49-a18b-b97592ac93d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.597668 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee7466-606b-48cb-a7f7-bf40c880777d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:22.599060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 775/1551] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.600209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b23b8-98f4-4f66-8a2f-8e3c3e8e26b7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.600497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b23b8-98f4-4f66-8a2f-8e3c3e8e26b7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.600784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b23b8-98f4-4f66-8a2f-8e3c3e8e26b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.606994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff7c3d-c57d-4b2f-957c-1a88a3d75f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:22.607464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff7c3d-c57d-4b2f-957c-1a88a3d75f47 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.607749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff7c3d-c57d-4b2f-957c-1a88a3d75f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.608027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff7c3d-c57d-4b2f-957c-1a88a3d75f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.608824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff7c3d-c57d-4b2f-957c-1a88a3d75f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:22.610402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff7c3d-c57d-4b2f-957c-1a88a3d75f47 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:22.620930 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2a4b23b8-98f4-4f66-8a2f-8e3c3e8e26b7 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.622097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 777/1552] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:22 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:22.623015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2a4b23b8-98f4-4f66-8a2f-8e3c3e8e26b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.623380 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08892-142b-4d39-94ef-1ca60744714c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.623675 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08892-142b-4d39-94ef-1ca60744714c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.623957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a08892-142b-4d39-94ef-1ca60744714c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.632202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 776/1553] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.634368 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fa8a7-b243-4acd-aaa6-9aa86930b183 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.634656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fa8a7-b243-4acd-aaa6-9aa86930b183 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.634961 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7fa8a7-b243-4acd-aaa6-9aa86930b183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.643732 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-71a08892-142b-4d39-94ef-1ca60744714c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.645173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-71a08892-142b-4d39-94ef-1ca60744714c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.653560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 778/1554] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.654356 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a08892-142b-4d39-94ef-1ca60744714c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:22.658045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9d7fa8a7-b243-4acd-aaa6-9aa86930b183 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.659359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9d7fa8a7-b243-4acd-aaa6-9aa86930b183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.663694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a08892-142b-4d39-94ef-1ca60744714c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:22.668267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 777/1555] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.669726 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693b353-99f4-42ef-b9ea-a8a448631c87 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.670036 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693b353-99f4-42ef-b9ea-a8a448631c87 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.670338 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693b353-99f4-42ef-b9ea-a8a448631c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.671143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ae3c3-9cce-472c-b47c-df0de1479c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:22.673535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ae3c3-9cce-472c-b47c-df0de1479c30 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.673535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ae3c3-9cce-472c-b47c-df0de1479c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.673535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ae3c3-9cce-472c-b47c-df0de1479c30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.673535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ae3c3-9cce-472c-b47c-df0de1479c30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:22.673894 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ae3c3-9cce-472c-b47c-df0de1479c30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:22.675855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ae3c3-9cce-472c-b47c-df0de1479c30 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:22.688732 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 779/1556] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:22.692363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4693b353-99f4-42ef-b9ea-a8a448631c87 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.693005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ae3c3-9cce-472c-b47c-df0de1479c30 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:22.693561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4693b353-99f4-42ef-b9ea-a8a448631c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.701606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 778/1557] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.703115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9902005-b912-42ce-8e61-b08453f9fa23 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.703442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9902005-b912-42ce-8e61-b08453f9fa23 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.703675 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9902005-b912-42ce-8e61-b08453f9fa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.704304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ae3c3-9cce-472c-b47c-df0de1479c30 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:22.709402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6fa2f-7f43-4e40-8afa-5173bf7b46f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:22.710041 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6fa2f-7f43-4e40-8afa-5173bf7b46f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.710322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6fa2f-7f43-4e40-8afa-5173bf7b46f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.710596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6fa2f-7f43-4e40-8afa-5173bf7b46f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.711299 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6fa2f-7f43-4e40-8afa-5173bf7b46f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:22.713242 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6fa2f-7f43-4e40-8afa-5173bf7b46f4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:22.723523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a9902005-b912-42ce-8e61-b08453f9fa23 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.724721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a9902005-b912-42ce-8e61-b08453f9fa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.733364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 779/1558] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.734482 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9902005-b912-42ce-8e61-b08453f9fa23 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:22.744014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9902005-b912-42ce-8e61-b08453f9fa23 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:22.748089 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 780/1559] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/projects => generated 388 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:22.748089 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed94fca-c6ce-44d8-a07e-31db2a2c37f6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.748089 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed94fca-c6ce-44d8-a07e-31db2a2c37f6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.748089 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed94fca-c6ce-44d8-a07e-31db2a2c37f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.766467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0ed94fca-c6ce-44d8-a07e-31db2a2c37f6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.767737 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0ed94fca-c6ce-44d8-a07e-31db2a2c37f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.809689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c6768-c5d9-4adf-9a40-1fb6b771bee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:22.810303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c6768-c5d9-4adf-9a40-1fb6b771bee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.810590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c6768-c5d9-4adf-9a40-1fb6b771bee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.810905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c6768-c5d9-4adf-9a40-1fb6b771bee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.811603 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c6768-c5d9-4adf-9a40-1fb6b771bee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:22.812632 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:22.813242 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c6768-c5d9-4adf-9a40-1fb6b771bee0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:22.826912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 780/1560] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:22 2022] GET /identity/v3/roles => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:22.826912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c6768-c5d9-4adf-9a40-1fb6b771bee0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:22.833053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 781/1561] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.834546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464910f-3fea-4e9a-aa03-49c1d2742f62 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.834864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464910f-3fea-4e9a-aa03-49c1d2742f62 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.835146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464910f-3fea-4e9a-aa03-49c1d2742f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.838564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c6768-c5d9-4adf-9a40-1fb6b771bee0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:22.843746 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9265b-358d-4d34-8519-d67520219d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:22.844313 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9265b-358d-4d34-8519-d67520219d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.844595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9265b-358d-4d34-8519-d67520219d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.844872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9265b-358d-4d34-8519-d67520219d7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.845570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9265b-358d-4d34-8519-d67520219d7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:22.845877 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9265b-358d-4d34-8519-d67520219d7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:22.846934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-d9f9265b-358d-4d34-8519-d67520219d7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:14:22.846934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Aug 01 15:14:22.846934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: Aug 01 15:14:22.847827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9265b-358d-4d34-8519-d67520219d7a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:22.855824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c464910f-3fea-4e9a-aa03-49c1d2742f62 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.857058 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c464910f-3fea-4e9a-aa03-49c1d2742f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.860795 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 781/1562] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:22.861402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9265b-358d-4d34-8519-d67520219d7a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:22.865694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 782/1563] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.866954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fbdd7-7367-4838-99ae-07da8cfa0b4d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.867249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fbdd7-7367-4838-99ae-07da8cfa0b4d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.868035 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834fbdd7-7367-4838-99ae-07da8cfa0b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.871765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9265b-358d-4d34-8519-d67520219d7a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:22.876473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5df61-fa46-437e-a797-8bf8aaadb05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:22.877075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5df61-fa46-437e-a797-8bf8aaadb05a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.877381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5df61-fa46-437e-a797-8bf8aaadb05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.877782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5df61-fa46-437e-a797-8bf8aaadb05a admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.878374 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5df61-fa46-437e-a797-8bf8aaadb05a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:22.880321 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5df61-fa46-437e-a797-8bf8aaadb05a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:22.887510 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-834fbdd7-7367-4838-99ae-07da8cfa0b4d None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:22.888714 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-834fbdd7-7367-4838-99ae-07da8cfa0b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:22.891910 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:22.897517 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 783/1564] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:22.899867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fbdd7-7367-4838-99ae-07da8cfa0b4d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:22.909889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834fbdd7-7367-4838-99ae-07da8cfa0b4d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:22.910559 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 782/1565] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/projects => generated 404 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:22.911387 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5df61-fa46-437e-a797-8bf8aaadb05a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:22.969031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b315f7b-cba6-4e4a-9d13-8523c40fc6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:22.969659 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b315f7b-cba6-4e4a-9d13-8523c40fc6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.969941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b315f7b-cba6-4e4a-9d13-8523c40fc6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.970217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b315f7b-cba6-4e4a-9d13-8523c40fc6e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.970948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b315f7b-cba6-4e4a-9d13-8523c40fc6e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:22.971257 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b315f7b-cba6-4e4a-9d13-8523c40fc6e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:22.972901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b315f7b-cba6-4e4a-9d13-8523c40fc6e9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:22.974448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5df61-fa46-437e-a797-8bf8aaadb05a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:22.979631 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3cd36-8625-4606-8c1f-42d0f0fb0b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:22.980796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3cd36-8625-4606-8c1f-42d0f0fb0b71 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:22.981078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3cd36-8625-4606-8c1f-42d0f0fb0b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:22.981352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3cd36-8625-4606-8c1f-42d0f0fb0b71 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:22.982044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3cd36-8625-4606-8c1f-42d0f0fb0b71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:22.983981 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3cd36-8625-4606-8c1f-42d0f0fb0b71 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:22.985049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 784/1566] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:22.987570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b315f7b-cba6-4e4a-9d13-8523c40fc6e9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:22.998231 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b315f7b-cba6-4e4a-9d13-8523c40fc6e9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.003307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe0e19-2486-46d3-8736-2d0bb4eedc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.004241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe0e19-2486-46d3-8736-2d0bb4eedc94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.004552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe0e19-2486-46d3-8736-2d0bb4eedc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.004985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe0e19-2486-46d3-8736-2d0bb4eedc94 admin admin] PATH_INFO: `/v3/projects/18f123f171554a58a2dea8fcc2afbefd/users/f33ea26c1f72445bb4cb50ea503600dc/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.006038 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 783/1567] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:22 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:23.009810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3cd36-8625-4606-8c1f-42d0f0fb0b71 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.020095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3cd36-8625-4606-8c1f-42d0f0fb0b71 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.020799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe0e19-2486-46d3-8736-2d0bb4eedc94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f123f171554a58a2dea8fcc2afbefd, user_id=f33ea26c1f72445bb4cb50ea503600dc, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.021885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-12fe0e19-2486-46d3-8736-2d0bb4eedc94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:14:23.021885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Aug 01 15:14:23.021885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: Aug 01 15:14:23.023933 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe0e19-2486-46d3-8736-2d0bb4eedc94 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.025179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabc5ea-a001-4b10-a911-88bc05a29b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.025876 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabc5ea-a001-4b10-a911-88bc05a29b27 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.026218 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabc5ea-a001-4b10-a911-88bc05a29b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.026615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabc5ea-a001-4b10-a911-88bc05a29b27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.028221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caabc5ea-a001-4b10-a911-88bc05a29b27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:23.028533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caabc5ea-a001-4b10-a911-88bc05a29b27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.030538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caabc5ea-a001-4b10-a911-88bc05a29b27 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.038324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 785/1568] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:22 2022] PUT /identity/v3/projects/18f123f171554a58a2dea8fcc2afbefd/users/f33ea26c1f72445bb4cb50ea503600dc/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:23.039367 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe0e19-2486-46d3-8736-2d0bb4eedc94 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.051529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 784/1569] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:23.051529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabc5ea-a001-4b10-a911-88bc05a29b27 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.056868 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabc5ea-a001-4b10-a911-88bc05a29b27 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.113913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f183a-3c4f-4c07-b862-287da7525440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.114185 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe0e19-2486-46d3-8736-2d0bb4eedc94 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.114695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f183a-3c4f-4c07-b862-287da7525440 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.114854 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f183a-3c4f-4c07-b862-287da7525440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.115131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f183a-3c4f-4c07-b862-287da7525440 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.115829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f183a-3c4f-4c07-b862-287da7525440 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:23.116168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f183a-3c4f-4c07-b862-287da7525440 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.117786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f183a-3c4f-4c07-b862-287da7525440 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.119386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b097d-e86e-496c-a5be-09b58331dcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.120073 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b097d-e86e-496c-a5be-09b58331dcd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.120354 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b097d-e86e-496c-a5be-09b58331dcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.120629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b097d-e86e-496c-a5be-09b58331dcd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.121814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b097d-e86e-496c-a5be-09b58331dcd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.123746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b097d-e86e-496c-a5be-09b58331dcd1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.130112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 785/1570] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:23.130893 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f183a-3c4f-4c07-b862-287da7525440 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.140613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f183a-3c4f-4c07-b862-287da7525440 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.145438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83093b7-a16b-42bb-92ec-1b9b6bab9b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.146168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83093b7-a16b-42bb-92ec-1b9b6bab9b76 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.146489 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83093b7-a16b-42bb-92ec-1b9b6bab9b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.146899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83093b7-a16b-42bb-92ec-1b9b6bab9b76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.147649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83093b7-a16b-42bb-92ec-1b9b6bab9b76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:23.148025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83093b7-a16b-42bb-92ec-1b9b6bab9b76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.149803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83093b7-a16b-42bb-92ec-1b9b6bab9b76 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.155769 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 786/1571] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:23 2022] POST /identity/v3/projects => generated 380 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:23.156653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b097d-e86e-496c-a5be-09b58331dcd1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.162577 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 786/1572] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:23.163687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83093b7-a16b-42bb-92ec-1b9b6bab9b76 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.179829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83093b7-a16b-42bb-92ec-1b9b6bab9b76 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.189720 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:23.221741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b097d-e86e-496c-a5be-09b58331dcd1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.226749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:23.226895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb80f01-7940-411b-b494-afbf415cc513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.227422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb80f01-7940-411b-b494-afbf415cc513 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.227705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb80f01-7940-411b-b494-afbf415cc513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.228004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb80f01-7940-411b-b494-afbf415cc513 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.228785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb80f01-7940-411b-b494-afbf415cc513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.231351 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb80f01-7940-411b-b494-afbf415cc513 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.243356 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b719d-62a1-4d81-a59a-0cc3dd91c801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.243988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b719d-62a1-4d81-a59a-0cc3dd91c801 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.244268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b719d-62a1-4d81-a59a-0cc3dd91c801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.244567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b719d-62a1-4d81-a59a-0cc3dd91c801 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.245278 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b719d-62a1-4d81-a59a-0cc3dd91c801 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.247220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b719d-62a1-4d81-a59a-0cc3dd91c801 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.253632 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 787/1573] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:23 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:23.254483 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb80f01-7940-411b-b494-afbf415cc513 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.265817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb80f01-7940-411b-b494-afbf415cc513 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.271305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d67664-4aad-4e40-ae1f-cd86cc08aea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.271920 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d67664-4aad-4e40-ae1f-cd86cc08aea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.272206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d67664-4aad-4e40-ae1f-cd86cc08aea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.272483 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d67664-4aad-4e40-ae1f-cd86cc08aea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.273174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d67664-4aad-4e40-ae1f-cd86cc08aea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.275122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d67664-4aad-4e40-ae1f-cd86cc08aea7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.279855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 787/1574] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:23 2022] POST /identity/v3/projects => generated 392 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:23.279855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263118e-2597-4e15-b543-cd07b26f7a43 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.279855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263118e-2597-4e15-b543-cd07b26f7a43 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.279855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263118e-2597-4e15-b543-cd07b26f7a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.290617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 788/1575] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:23 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:23.291748 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d67664-4aad-4e40-ae1f-cd86cc08aea7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.299200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a263118e-2597-4e15-b543-cd07b26f7a43 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:23.300405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a263118e-2597-4e15-b543-cd07b26f7a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:23.301347 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d67664-4aad-4e40-ae1f-cd86cc08aea7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.360581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d7fff-68a7-4134-8c64-b4a6b0b81f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.361209 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d7fff-68a7-4134-8c64-b4a6b0b81f8f admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.361493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d7fff-68a7-4134-8c64-b4a6b0b81f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.361603 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 788/1576] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:23.361792 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d7fff-68a7-4134-8c64-b4a6b0b81f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.362404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263118e-2597-4e15-b543-cd07b26f7a43 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.362529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d7fff-68a7-4134-8c64-b4a6b0b81f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.366043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d7fff-68a7-4134-8c64-b4a6b0b81f8f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.372069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263118e-2597-4e15-b543-cd07b26f7a43 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.376616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7b3b6-50fa-4bea-8e56-c4232929c2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.377273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7b3b6-50fa-4bea-8e56-c4232929c2be admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.377554 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7b3b6-50fa-4bea-8e56-c4232929c2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.377831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7b3b6-50fa-4bea-8e56-c4232929c2be admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.378499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f7b3b6-50fa-4bea-8e56-c4232929c2be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.380442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f7b3b6-50fa-4bea-8e56-c4232929c2be admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.395377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 789/1577] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:23 2022] POST /identity/v3/projects => generated 366 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:23.396281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d7fff-68a7-4134-8c64-b4a6b0b81f8f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.413930 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 789/1578] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:23 2022] POST /identity/v3/projects => generated 374 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:23.413930 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7b3b6-50fa-4bea-8e56-c4232929c2be admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.462505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d7fff-68a7-4134-8c64-b4a6b0b81f8f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.475137 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7b3b6-50fa-4bea-8e56-c4232929c2be admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.479886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e33726-fa84-4734-aff7-25d030d50091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.480536 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e33726-fa84-4734-aff7-25d030d50091 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.480821 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e33726-fa84-4734-aff7-25d030d50091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.481093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e33726-fa84-4734-aff7-25d030d50091 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.481766 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e33726-fa84-4734-aff7-25d030d50091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.483732 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e33726-fa84-4734-aff7-25d030d50091 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.501042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:23.513438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 790/1579] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:23 2022] POST /identity/v3/projects => generated 372 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:23.517574 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e33726-fa84-4734-aff7-25d030d50091 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.521246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7148239-aa81-4fd6-af39-10e57724c46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.521876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7148239-aa81-4fd6-af39-10e57724c46a admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.522158 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7148239-aa81-4fd6-af39-10e57724c46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.522432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7148239-aa81-4fd6-af39-10e57724c46a admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.523141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7148239-aa81-4fd6-af39-10e57724c46a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.525088 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7148239-aa81-4fd6-af39-10e57724c46a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.554158 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 790/1580] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:23 2022] POST /identity/v3/users => generated 291 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:23.557559 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7148239-aa81-4fd6-af39-10e57724c46a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.561911 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:23.568440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7148239-aa81-4fd6-af39-10e57724c46a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.583599 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e33726-fa84-4734-aff7-25d030d50091 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.588334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23aad1-0cb7-4c54-bb12-a91a7ba652c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.589077 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23aad1-0cb7-4c54-bb12-a91a7ba652c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.589361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23aad1-0cb7-4c54-bb12-a91a7ba652c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.590372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23aad1-0cb7-4c54-bb12-a91a7ba652c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.591100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23aad1-0cb7-4c54-bb12-a91a7ba652c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.592696 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23aad1-0cb7-4c54-bb12-a91a7ba652c7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.602944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 791/1581] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:23 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:23.603968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23aad1-0cb7-4c54-bb12-a91a7ba652c7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.613899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23aad1-0cb7-4c54-bb12-a91a7ba652c7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.618611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf657e-86d5-456d-a4a9-6fded29b5805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.619186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf657e-86d5-456d-a4a9-6fded29b5805 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.619470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf657e-86d5-456d-a4a9-6fded29b5805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.619751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf657e-86d5-456d-a4a9-6fded29b5805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.620435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf657e-86d5-456d-a4a9-6fded29b5805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.622692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf657e-86d5-456d-a4a9-6fded29b5805 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.629324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888cbdc-4ba8-4b44-85e4-801efc10bf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.629978 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888cbdc-4ba8-4b44-85e4-801efc10bf2a admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.630257 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888cbdc-4ba8-4b44-85e4-801efc10bf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.630531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888cbdc-4ba8-4b44-85e4-801efc10bf2a admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.631228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888cbdc-4ba8-4b44-85e4-801efc10bf2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.632950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 792/1582] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:23 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:23.633265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888cbdc-4ba8-4b44-85e4-801efc10bf2a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.633755 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf657e-86d5-456d-a4a9-6fded29b5805 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.658631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 791/1583] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:23 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:23.658631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888cbdc-4ba8-4b44-85e4-801efc10bf2a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.668921 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888cbdc-4ba8-4b44-85e4-801efc10bf2a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.674276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc78ce-9350-4b57-8915-4118d756d27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.674925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc78ce-9350-4b57-8915-4118d756d27e admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.675214 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc78ce-9350-4b57-8915-4118d756d27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.675492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc78ce-9350-4b57-8915-4118d756d27e admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.676194 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc78ce-9350-4b57-8915-4118d756d27e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.678283 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc78ce-9350-4b57-8915-4118d756d27e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.694610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf657e-86d5-456d-a4a9-6fded29b5805 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.698402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc68b9-9203-4858-8c62-4782075b64c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.699154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc68b9-9203-4858-8c62-4782075b64c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.699443 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc68b9-9203-4858-8c62-4782075b64c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.699719 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dc68b9-9203-4858-8c62-4782075b64c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.699881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 792/1584] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:23 2022] POST /identity/v3/users => generated 284 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:23.700695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc78ce-9350-4b57-8915-4118d756d27e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.712068 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc68b9-9203-4858-8c62-4782075b64c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.712532 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc78ce-9350-4b57-8915-4118d756d27e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.713523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dc68b9-9203-4858-8c62-4782075b64c0 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.718570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe97c60-41f7-4808-bf0c-230615cf1f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.719495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe97c60-41f7-4808-bf0c-230615cf1f1e admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.719986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe97c60-41f7-4808-bf0c-230615cf1f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.720474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe97c60-41f7-4808-bf0c-230615cf1f1e admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.721641 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe97c60-41f7-4808-bf0c-230615cf1f1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.724461 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe97c60-41f7-4808-bf0c-230615cf1f1e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.725325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 793/1585] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:23.726301 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc68b9-9203-4858-8c62-4782075b64c0 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.736833 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dc68b9-9203-4858-8c62-4782075b64c0 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.742346 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c96f8-acca-4049-a63d-74319452969f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.743035 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c96f8-acca-4049-a63d-74319452969f admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.743286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c96f8-acca-4049-a63d-74319452969f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.743667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c96f8-acca-4049-a63d-74319452969f admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.744360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c96f8-acca-4049-a63d-74319452969f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.747372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c96f8-acca-4049-a63d-74319452969f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.759419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 793/1586] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:23 2022] POST /identity/v3/users => generated 288 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:23.760181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe97c60-41f7-4808-bf0c-230615cf1f1e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.771554 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe97c60-41f7-4808-bf0c-230615cf1f1e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.776293 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fa507-1d71-472f-8812-304b4c68d8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.780456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fa507-1d71-472f-8812-304b4c68d8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.780639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 794/1587] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:23 2022] POST /identity/v3/users => generated 287 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:23.780775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fa507-1d71-472f-8812-304b4c68d8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.780775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fa507-1d71-472f-8812-304b4c68d8e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.780964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c96f8-acca-4049-a63d-74319452969f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.781051 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fa507-1d71-472f-8812-304b4c68d8e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.781051 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fa507-1d71-472f-8812-304b4c68d8e3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.793525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 794/1588] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:23 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:23.794772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fa507-1d71-472f-8812-304b4c68d8e3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.794904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c96f8-acca-4049-a63d-74319452969f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.799754 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db935d7-d589-4a54-a243-c401ac7cf501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.801147 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db935d7-d589-4a54-a243-c401ac7cf501 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.801429 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db935d7-d589-4a54-a243-c401ac7cf501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.801707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db935d7-d589-4a54-a243-c401ac7cf501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.802986 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db935d7-d589-4a54-a243-c401ac7cf501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.805024 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db935d7-d589-4a54-a243-c401ac7cf501 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.806504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fa507-1d71-472f-8812-304b4c68d8e3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.811810 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822b417-73c4-4a2e-b189-ea8fd421b5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.812565 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822b417-73c4-4a2e-b189-ea8fd421b5ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.812845 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822b417-73c4-4a2e-b189-ea8fd421b5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.813123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822b417-73c4-4a2e-b189-ea8fd421b5ad admin admin] PATH_INFO: `/v3/projects/2bb9ac7a058f47f39a01b9bd92c7f4a2/users/424ccf230e16423a8dd32097d90c4995/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.819778 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 795/1589] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:23 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:23.820466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db935d7-d589-4a54-a243-c401ac7cf501 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.831571 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822b417-73c4-4a2e-b189-ea8fd421b5ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bb9ac7a058f47f39a01b9bd92c7f4a2, user_id=424ccf230e16423a8dd32097d90c4995, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.834123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db935d7-d589-4a54-a243-c401ac7cf501 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.835046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822b417-73c4-4a2e-b189-ea8fd421b5ad admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.840217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84844592-acf0-4a26-982a-7f265fc16f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.841258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84844592-acf0-4a26-982a-7f265fc16f90 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.841904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84844592-acf0-4a26-982a-7f265fc16f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.842216 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84844592-acf0-4a26-982a-7f265fc16f90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.843901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84844592-acf0-4a26-982a-7f265fc16f90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.845246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84844592-acf0-4a26-982a-7f265fc16f90 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.854556 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 795/1590] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:23 2022] PUT /identity/v3/projects/2bb9ac7a058f47f39a01b9bd92c7f4a2/users/424ccf230e16423a8dd32097d90c4995/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:23.856291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822b417-73c4-4a2e-b189-ea8fd421b5ad admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.860306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 796/1591] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:23 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:23.861527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84844592-acf0-4a26-982a-7f265fc16f90 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.873462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84844592-acf0-4a26-982a-7f265fc16f90 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.915984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:23.938407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822b417-73c4-4a2e-b189-ea8fd421b5ad admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.944723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085433c-610b-41bb-a83b-87facb72e6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.945544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085433c-610b-41bb-a83b-87facb72e6aa admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.945827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085433c-610b-41bb-a83b-87facb72e6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.946106 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085433c-610b-41bb-a83b-87facb72e6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.946289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9672721-7cc5-461a-a134-b73ed465f03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.946949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9672721-7cc5-461a-a134-b73ed465f03a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.947069 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085433c-610b-41bb-a83b-87facb72e6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.947235 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9672721-7cc5-461a-a134-b73ed465f03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.947507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9672721-7cc5-461a-a134-b73ed465f03a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.948212 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9672721-7cc5-461a-a134-b73ed465f03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.948718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085433c-610b-41bb-a83b-87facb72e6aa admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.949880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9672721-7cc5-461a-a134-b73ed465f03a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.960949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 796/1592] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:23 2022] GET /identity/v3/roles => generated 1427 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:23.961884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085433c-610b-41bb-a83b-87facb72e6aa admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.962204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 797/1593] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:23 2022] GET /identity/v3/roles => generated 1427 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:23.963540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9672721-7cc5-461a-a134-b73ed465f03a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:23.974163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085433c-610b-41bb-a83b-87facb72e6aa admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.975159 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9672721-7cc5-461a-a134-b73ed465f03a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:23.979146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533b95d-a77a-4654-8da7-02587d4aef83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.979768 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533b95d-a77a-4654-8da7-02587d4aef83 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.980138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533b95d-a77a-4654-8da7-02587d4aef83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.980831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723fe87-4f94-40a7-93a8-b096f51afcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:23.980958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533b95d-a77a-4654-8da7-02587d4aef83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.981421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723fe87-4f94-40a7-93a8-b096f51afcef admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:23.981591 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533b95d-a77a-4654-8da7-02587d4aef83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.981688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723fe87-4f94-40a7-93a8-b096f51afcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:23.982005 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723fe87-4f94-40a7-93a8-b096f51afcef admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:23.982725 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723fe87-4f94-40a7-93a8-b096f51afcef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:23.983230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3533b95d-a77a-4654-8da7-02587d4aef83 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.984461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723fe87-4f94-40a7-93a8-b096f51afcef admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:23.995716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 797/1594] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:23 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:23.996383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533b95d-a77a-4654-8da7-02587d4aef83 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:24.003002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 798/1595] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:23 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:24.005364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3323-a8df-4076-a9ff-8ed7ebfcf68c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.005744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3323-a8df-4076-a9ff-8ed7ebfcf68c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.006192 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3323-a8df-4076-a9ff-8ed7ebfcf68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.008235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533b95d-a77a-4654-8da7-02587d4aef83 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:24.014498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bffe4-ee60-47ca-938b-3d91034de60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:24.015504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bffe4-ee60-47ca-938b-3d91034de60d admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.015793 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bffe4-ee60-47ca-938b-3d91034de60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.016118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69bffe4-ee60-47ca-938b-3d91034de60d admin admin] PATH_INFO: `/v3/projects/7517df1bfbef478a83f2a55e2a340799/users/ea431174273e48e5bdd7e2ab0a03f14f/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.028276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bdfc3323-a8df-4076-a9ff-8ed7ebfcf68c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:24.030017 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bdfc3323-a8df-4076-a9ff-8ed7ebfcf68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:24.034238 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bffe4-ee60-47ca-938b-3d91034de60d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7517df1bfbef478a83f2a55e2a340799, user_id=ea431174273e48e5bdd7e2ab0a03f14f, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:24.037175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69bffe4-ee60-47ca-938b-3d91034de60d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:24.041827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 799/1596] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:24.041827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3323-a8df-4076-a9ff-8ed7ebfcf68c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:24.050088 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:24.054302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 798/1597] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:23 2022] PUT /identity/v3/projects/7517df1bfbef478a83f2a55e2a340799/users/ea431174273e48e5bdd7e2ab0a03f14f/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:24.055627 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3323-a8df-4076-a9ff-8ed7ebfcf68c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:24.058810 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bffe4-ee60-47ca-938b-3d91034de60d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:24.129969 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69bffe4-ee60-47ca-938b-3d91034de60d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:24.135342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5fe307-d383-429b-8742-faef7ce487dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:24.136026 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fe307-d383-429b-8742-faef7ce487dc admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.136843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fe307-d383-429b-8742-faef7ce487dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.137219 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fe307-d383-429b-8742-faef7ce487dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.137933 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5fe307-d383-429b-8742-faef7ce487dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:24.139665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5fe307-d383-429b-8742-faef7ce487dc admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:24.151387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 799/1598] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:24 2022] GET /identity/v3/roles => generated 1427 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:24.152834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5fe307-d383-429b-8742-faef7ce487dc admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:24.164535 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0a65e-9214-4cb6-9830-c9573fd54bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:24.165108 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0a65e-9214-4cb6-9830-c9573fd54bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.165220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0a65e-9214-4cb6-9830-c9573fd54bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.165456 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba0a65e-9214-4cb6-9830-c9573fd54bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.165658 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5fe307-d383-429b-8742-faef7ce487dc admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:24.166110 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0a65e-9214-4cb6-9830-c9573fd54bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:24.168336 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba0a65e-9214-4cb6-9830-c9573fd54bf9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:24.170975 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef91c83-ecd8-4946-9183-542a579fc0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:24.171948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef91c83-ecd8-4946-9183-542a579fc0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.172240 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef91c83-ecd8-4946-9183-542a579fc0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.172543 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef91c83-ecd8-4946-9183-542a579fc0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.173256 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef91c83-ecd8-4946-9183-542a579fc0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:24.174985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef91c83-ecd8-4946-9183-542a579fc0d3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:24.185112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 800/1599] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:24 2022] GET /identity/v3/roles => generated 1427 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:24.186099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0a65e-9214-4cb6-9830-c9573fd54bf9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:24.191200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 800/1600] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:24 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:24.191200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef91c83-ecd8-4946-9183-542a579fc0d3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:24.198338 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba0a65e-9214-4cb6-9830-c9573fd54bf9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:24.202200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef91c83-ecd8-4946-9183-542a579fc0d3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:24.203892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e10fbb-69ec-4b77-bb38-bda123d6ce8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:24.204576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e10fbb-69ec-4b77-bb38-bda123d6ce8a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.204959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e10fbb-69ec-4b77-bb38-bda123d6ce8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.205232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e10fbb-69ec-4b77-bb38-bda123d6ce8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.205938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e10fbb-69ec-4b77-bb38-bda123d6ce8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:24.208105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e10fbb-69ec-4b77-bb38-bda123d6ce8a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:24.208746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a64f6-c23c-4c9f-afbd-ed4898b0e88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:24.210087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a64f6-c23c-4c9f-afbd-ed4898b0e88c admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.210449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a64f6-c23c-4c9f-afbd-ed4898b0e88c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.210885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a64f6-c23c-4c9f-afbd-ed4898b0e88c admin admin] PATH_INFO: `/v3/projects/85a7e64a4f0a43599f9eeeb99e695f24/users/e8fc60410fab43a39bdb80478348ddae/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.231233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a64f6-c23c-4c9f-afbd-ed4898b0e88c admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a7e64a4f0a43599f9eeeb99e695f24, user_id=e8fc60410fab43a39bdb80478348ddae, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:24.231844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 801/1601] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:24 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:24.234145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a64f6-c23c-4c9f-afbd-ed4898b0e88c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:24.236599 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e10fbb-69ec-4b77-bb38-bda123d6ce8a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:24.252247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 801/1602] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:24 2022] PUT /identity/v3/projects/85a7e64a4f0a43599f9eeeb99e695f24/users/e8fc60410fab43a39bdb80478348ddae/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:24.253929 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd73eb7-de52-4b72-9ed7-2fd38bb12f5f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.254272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd73eb7-de52-4b72-9ed7-2fd38bb12f5f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.254608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd73eb7-de52-4b72-9ed7-2fd38bb12f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.277538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5cd73eb7-de52-4b72-9ed7-2fd38bb12f5f None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:24.278921 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5cd73eb7-de52-4b72-9ed7-2fd38bb12f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:24.301759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e10fbb-69ec-4b77-bb38-bda123d6ce8a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:24.362872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 802/1603] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:24.363114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43712b86-e4ac-481e-b0f2-1fa208144396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:24.363615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43712b86-e4ac-481e-b0f2-1fa208144396 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.363917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43712b86-e4ac-481e-b0f2-1fa208144396 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.364598 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43712b86-e4ac-481e-b0f2-1fa208144396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.367458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd73eb7-de52-4b72-9ed7-2fd38bb12f5f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:24.377036 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43712b86-e4ac-481e-b0f2-1fa208144396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:24.378485 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43712b86-e4ac-481e-b0f2-1fa208144396 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:24.381043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd73eb7-de52-4b72-9ed7-2fd38bb12f5f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:24.388898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbf186-23d3-4d08-835c-e5b4437886bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:24.390070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbf186-23d3-4d08-835c-e5b4437886bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.390209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 802/1604] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:24.390678 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbf186-23d3-4d08-835c-e5b4437886bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.390949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43712b86-e4ac-481e-b0f2-1fa208144396 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:24.391265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fbf186-23d3-4d08-835c-e5b4437886bc admin admin] PATH_INFO: `/v3/projects/0e3cc1543ceb45bf97c8547a648f6e65/users/7f811f71df43405b81e27cfc6045f1cf/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.400460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:24.400605 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43712b86-e4ac-481e-b0f2-1fa208144396 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:24.406689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effac70a-ccbb-48db-aff3-1c1aebe4f819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:24.408145 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effac70a-ccbb-48db-aff3-1c1aebe4f819 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.408145 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effac70a-ccbb-48db-aff3-1c1aebe4f819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.408343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fbf186-23d3-4d08-835c-e5b4437886bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e3cc1543ceb45bf97c8547a648f6e65, user_id=7f811f71df43405b81e27cfc6045f1cf, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:24.408447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effac70a-ccbb-48db-aff3-1c1aebe4f819 admin admin] PATH_INFO: `/v3/projects/e5e42e0f0ef74615ac27486ada272fd8/users/ca1d32ee23bc42b38ea9a043f7a850f5/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.411058 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fbf186-23d3-4d08-835c-e5b4437886bc admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:24.427708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 803/1605] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:24 2022] PUT /identity/v3/projects/0e3cc1543ceb45bf97c8547a648f6e65/users/7f811f71df43405b81e27cfc6045f1cf/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:24.428830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbf186-23d3-4d08-835c-e5b4437886bc admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:24.436402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effac70a-ccbb-48db-aff3-1c1aebe4f819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e42e0f0ef74615ac27486ada272fd8, user_id=ca1d32ee23bc42b38ea9a043f7a850f5, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:24.439540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effac70a-ccbb-48db-aff3-1c1aebe4f819 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:24.462944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 803/1606] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:24 2022] PUT /identity/v3/projects/e5e42e0f0ef74615ac27486ada272fd8/users/ca1d32ee23bc42b38ea9a043f7a850f5/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:24.463800 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effac70a-ccbb-48db-aff3-1c1aebe4f819 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:24.508076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:24.522445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fbf186-23d3-4d08-835c-e5b4437886bc admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:24.572479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effac70a-ccbb-48db-aff3-1c1aebe4f819 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:24.583014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b600c-c38d-4746-a6a0-a82c10fa3317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:24.584371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b600c-c38d-4746-a6a0-a82c10fa3317 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.585270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b600c-c38d-4746-a6a0-a82c10fa3317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.585884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b600c-c38d-4746-a6a0-a82c10fa3317 admin admin] PATH_INFO: `/v3/projects/b7d2693930994c6c990461c488626517/users/47f5aec582c742b18cb17bddd77f2891/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.610163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b600c-c38d-4746-a6a0-a82c10fa3317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d2693930994c6c990461c488626517, user_id=47f5aec582c742b18cb17bddd77f2891, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:24.614398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b600c-c38d-4746-a6a0-a82c10fa3317 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:24.630954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43be43-663f-4709-bf43-f41101d30072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:24.632012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43be43-663f-4709-bf43-f41101d30072 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.632352 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43be43-663f-4709-bf43-f41101d30072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.632628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43be43-663f-4709-bf43-f41101d30072 admin admin] PATH_INFO: `/v3/projects/696478725b424726a5c81d6ea974f819/users/aed8ce0800f8449fb1319bcb7ee3d9e5/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.636075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 804/1607] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:24 2022] PUT /identity/v3/projects/b7d2693930994c6c990461c488626517/users/47f5aec582c742b18cb17bddd77f2891/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:24.638847 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75ed79-16c7-4d54-970c-d612fbef3710 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.639211 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75ed79-16c7-4d54-970c-d612fbef3710 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.639544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75ed79-16c7-4d54-970c-d612fbef3710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.651747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43be43-663f-4709-bf43-f41101d30072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=696478725b424726a5c81d6ea974f819, user_id=aed8ce0800f8449fb1319bcb7ee3d9e5, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:24.655256 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43be43-663f-4709-bf43-f41101d30072 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:24.660020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ce75ed79-16c7-4d54-970c-d612fbef3710 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:24.661256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ce75ed79-16c7-4d54-970c-d612fbef3710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:24.671253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 804/1608] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:24 2022] PUT /identity/v3/projects/696478725b424726a5c81d6ea974f819/users/aed8ce0800f8449fb1319bcb7ee3d9e5/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:24.672003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43be43-663f-4709-bf43-f41101d30072 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:24.794633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43be43-663f-4709-bf43-f41101d30072 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:24.796233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 805/1609] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:24.800613 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa6077-2b69-45b5-b0cc-7b62f77c2ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:24.801405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a320e9-a0db-47ea-a4eb-175197303ba0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.801530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa6077-2b69-45b5-b0cc-7b62f77c2ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.801757 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa6077-2b69-45b5-b0cc-7b62f77c2ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.801879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a320e9-a0db-47ea-a4eb-175197303ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.802118 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a320e9-a0db-47ea-a4eb-175197303ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.802239 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa6077-2b69-45b5-b0cc-7b62f77c2ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.842826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:24.853259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e2a320e9-a0db-47ea-a4eb-175197303ba0 None None] MFA Rules not processed for user `7f811f71df43405b81e27cfc6045f1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:24.858612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e2a320e9-a0db-47ea-a4eb-175197303ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:24.893480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa6077-2b69-45b5-b0cc-7b62f77c2ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:24.895052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa6077-2b69-45b5-b0cc-7b62f77c2ba9 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:24.921460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 805/1610] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:24.923183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f7b3b-69f9-4809-90f6-50c0208a68c1 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.923477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f7b3b-69f9-4809-90f6-50c0208a68c1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.923758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f7b3b-69f9-4809-90f6-50c0208a68c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.967974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 806/1611] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:24.969895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3045d-3f48-4a48-b1a8-563f74efc901 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.970604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3045d-3f48-4a48-b1a8-563f74efc901 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.971051 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3045d-3f48-4a48-b1a8-563f74efc901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:24.974982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-fc3f7b3b-69f9-4809-90f6-50c0208a68c1 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:24.976372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fc3f7b3b-69f9-4809-90f6-50c0208a68c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:24.986753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 806/1612] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:24.986753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b4384-f386-48cb-8990-86faf66857be None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:24.986753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b4384-f386-48cb-8990-86faf66857be None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:24.986753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b4384-f386-48cb-8990-86faf66857be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:25.022960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f58b4384-f386-48cb-8990-86faf66857be None None] MFA Rules not processed for user `aed8ce0800f8449fb1319bcb7ee3d9e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:25.023225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fbd3045d-3f48-4a48-b1a8-563f74efc901 None None] MFA Rules not processed for user `47f5aec582c742b18cb17bddd77f2891`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:25.024982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f58b4384-f386-48cb-8990-86faf66857be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:25.025443 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fbd3045d-3f48-4a48-b1a8-563f74efc901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:25.075564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:25.116750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 807/1613] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:25.117762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b4384-f386-48cb-8990-86faf66857be None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:25.139037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 807/1614] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:25.141089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3045d-3f48-4a48-b1a8-563f74efc901 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:25.154180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b4384-f386-48cb-8990-86faf66857be None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:25.157612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3045d-3f48-4a48-b1a8-563f74efc901 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:25.160678 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903af1d-5fae-434f-982d-222e70b79080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:25.161323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903af1d-5fae-434f-982d-222e70b79080 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:25.161612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903af1d-5fae-434f-982d-222e70b79080 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:25.161890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903af1d-5fae-434f-982d-222e70b79080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:25.164883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496438e2-7e33-4347-8564-2b6658ee802f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:25.165617 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496438e2-7e33-4347-8564-2b6658ee802f admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:25.166084 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496438e2-7e33-4347-8564-2b6658ee802f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:25.166471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496438e2-7e33-4347-8564-2b6658ee802f admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:25.167312 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496438e2-7e33-4347-8564-2b6658ee802f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:25.172132 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496438e2-7e33-4347-8564-2b6658ee802f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:25.188707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903af1d-5fae-434f-982d-222e70b79080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:25.193305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903af1d-5fae-434f-982d-222e70b79080 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:25.210859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 808/1615] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:25.211736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903af1d-5fae-434f-982d-222e70b79080 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:25.217420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 808/1616] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:25 2022] POST /identity/v3/projects => generated 364 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:25.218395 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5f0de-006e-4782-a60c-f0f87cbbd23e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:25.218821 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5f0de-006e-4782-a60c-f0f87cbbd23e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:25.219228 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5f0de-006e-4782-a60c-f0f87cbbd23e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:25.224110 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 809/1617] 10.222.0.100 () {54 vars in 967 bytes} [Mon Aug 1 15:14:25 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:14:25.226153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd5b24-792c-44b9-8310-c9f51b88b7a2 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:25.226542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd5b24-792c-44b9-8310-c9f51b88b7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:25.227295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd5b24-792c-44b9-8310-c9f51b88b7a2 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:25.229644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 810/1618] 10.222.0.100 () {54 vars in 967 bytes} [Mon Aug 1 15:14:25 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:14:25.231056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd5b24-792c-44b9-8310-c9f51b88b7a2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:25.231842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903af1d-5fae-434f-982d-222e70b79080 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:25.317773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60719d-0c84-42e9-b16c-ffca7a7e778d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:25.318418 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60719d-0c84-42e9-b16c-ffca7a7e778d service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:25.318793 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60719d-0c84-42e9-b16c-ffca7a7e778d service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:25.319087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60719d-0c84-42e9-b16c-ffca7a7e778d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:25.364290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd5b24-792c-44b9-8310-c9f51b88b7a2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:25.371008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6b70a-dac9-47ab-8340-ec09f52be54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:25.371774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6b70a-dac9-47ab-8340-ec09f52be54d admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:25.371993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6b70a-dac9-47ab-8340-ec09f52be54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:25.372355 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6b70a-dac9-47ab-8340-ec09f52be54d admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:25.374706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6b70a-dac9-47ab-8340-ec09f52be54d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:25.375696 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6b70a-dac9-47ab-8340-ec09f52be54d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:25.413122 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 811/1619] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:25 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:25.419031 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c785f46-d71e-4aec-a294-bbb13654d610 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:25.419340 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c785f46-d71e-4aec-a294-bbb13654d610 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:25.419621 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c785f46-d71e-4aec-a294-bbb13654d610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:25.436982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:25.455670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60719d-0c84-42e9-b16c-ffca7a7e778d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:25.457327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60719d-0c84-42e9-b16c-ffca7a7e778d service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:25.473705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 809/1620] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:25.475885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e7649-7904-486f-95c3-57d68dcfd8c4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:25.476243 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e7649-7904-486f-95c3-57d68dcfd8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:25.476592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e7649-7904-486f-95c3-57d68dcfd8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:25.478021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9c785f46-d71e-4aec-a294-bbb13654d610 None None] MFA Rules not processed for user `ec50ac7c54324884a1b2a6c63d617907`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:25.479321 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9c785f46-d71e-4aec-a294-bbb13654d610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:25.506541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-113e7649-7904-486f-95c3-57d68dcfd8c4 None None] MFA Rules not processed for user `ec50ac7c54324884a1b2a6c63d617907`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:25.510545 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-113e7649-7904-486f-95c3-57d68dcfd8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:25.636698 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 812/1621] 10.222.0.100 () {58 vars in 1064 bytes} [Mon Aug 1 15:14:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:25.638113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c785f46-d71e-4aec-a294-bbb13654d610 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:25.651143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c785f46-d71e-4aec-a294-bbb13654d610 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:25.663477 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88f19d-da73-43a1-8e74-1c8ad7ca9c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:25.665688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88f19d-da73-43a1-8e74-1c8ad7ca9c35 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:25.666078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88f19d-da73-43a1-8e74-1c8ad7ca9c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:25.666457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88f19d-da73-43a1-8e74-1c8ad7ca9c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:25.671778 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88f19d-da73-43a1-8e74-1c8ad7ca9c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:25.675732 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88f19d-da73-43a1-8e74-1c8ad7ca9c35 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:25.694751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 813/1622] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:25 2022] GET /identity/v3/roles => generated 1427 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:25.694751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88f19d-da73-43a1-8e74-1c8ad7ca9c35 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:25.712122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 810/1623] 10.222.0.100 () {58 vars in 1064 bytes} [Mon Aug 1 15:14:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:25.716563 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e7649-7904-486f-95c3-57d68dcfd8c4 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:25.723414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:25.740057 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e7649-7904-486f-95c3-57d68dcfd8c4 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:25.740487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88f19d-da73-43a1-8e74-1c8ad7ca9c35 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:25.747715 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02364f52-50e6-41b3-98d5-595955f1dd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:25.749979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02364f52-50e6-41b3-98d5-595955f1dd1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:25.750402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02364f52-50e6-41b3-98d5-595955f1dd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:25.750623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02364f52-50e6-41b3-98d5-595955f1dd1c admin admin] PATH_INFO: `/v3/projects/743e5b018819415bbd5548d6df4351e1/users/dd113de2c2604932b9cee204da77e5f9/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:25.751176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca604497-d928-4979-891f-aef2de6deeb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:25.752046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca604497-d928-4979-891f-aef2de6deeb3 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:25.752524 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca604497-d928-4979-891f-aef2de6deeb3 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:25.753006 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca604497-d928-4979-891f-aef2de6deeb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:25.777888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02364f52-50e6-41b3-98d5-595955f1dd1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=743e5b018819415bbd5548d6df4351e1, user_id=dd113de2c2604932b9cee204da77e5f9, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:25.783772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02364f52-50e6-41b3-98d5-595955f1dd1c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:25.823311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 811/1624] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:25 2022] PUT /identity/v3/projects/743e5b018819415bbd5548d6df4351e1/users/dd113de2c2604932b9cee204da77e5f9/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:25.832137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02364f52-50e6-41b3-98d5-595955f1dd1c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:25.876304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca604497-d928-4979-891f-aef2de6deeb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:25.877925 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca604497-d928-4979-891f-aef2de6deeb3 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:25.949486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 814/1625] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:25.949486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca604497-d928-4979-891f-aef2de6deeb3 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:25.949486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca604497-d928-4979-891f-aef2de6deeb3 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:26.024054 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02364f52-50e6-41b3-98d5-595955f1dd1c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:26.062493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0277d04f-7444-4182-9d7e-5f5410a3e431 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:26.063188 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277d04f-7444-4182-9d7e-5f5410a3e431 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:26.063471 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277d04f-7444-4182-9d7e-5f5410a3e431 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:26.063748 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0277d04f-7444-4182-9d7e-5f5410a3e431 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:26.099599 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5fdcb-7502-4eb2-a451-6adb46d8d05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:26.099912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5fdcb-7502-4eb2-a451-6adb46d8d05a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:26.100417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5fdcb-7502-4eb2-a451-6adb46d8d05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:26.100761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5fdcb-7502-4eb2-a451-6adb46d8d05a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:26.101485 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5fdcb-7502-4eb2-a451-6adb46d8d05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:26.108170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5fdcb-7502-4eb2-a451-6adb46d8d05a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:26.129963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 815/1626] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:25 2022] GET /identity/v3/roles => generated 1427 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:26.133813 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5fdcb-7502-4eb2-a451-6adb46d8d05a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:26.167836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5fdcb-7502-4eb2-a451-6adb46d8d05a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:26.173217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c286f19-0cfa-47b8-b494-87bb959d0506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:26.173968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c286f19-0cfa-47b8-b494-87bb959d0506 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:26.174255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c286f19-0cfa-47b8-b494-87bb959d0506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:26.174754 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c286f19-0cfa-47b8-b494-87bb959d0506 admin admin] PATH_INFO: `/v3/projects/743e5b018819415bbd5548d6df4351e1/users/dd113de2c2604932b9cee204da77e5f9/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:26.176766 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c286f19-0cfa-47b8-b494-87bb959d0506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=743e5b018819415bbd5548d6df4351e1, user_id=dd113de2c2604932b9cee204da77e5f9, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:26.179745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c286f19-0cfa-47b8-b494-87bb959d0506 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:26.211163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 816/1627] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:26 2022] PUT /identity/v3/projects/743e5b018819415bbd5548d6df4351e1/users/dd113de2c2604932b9cee204da77e5f9/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:14:26.221840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02d508-3d9e-4a21-8db3-2be7a67d2d92 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:26.222348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02d508-3d9e-4a21-8db3-2be7a67d2d92 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:26.222639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02d508-3d9e-4a21-8db3-2be7a67d2d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:26.249754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:26.266172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2b02d508-3d9e-4a21-8db3-2be7a67d2d92 None None] MFA Rules not processed for user `dd113de2c2604932b9cee204da77e5f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:26.269970 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2b02d508-3d9e-4a21-8db3-2be7a67d2d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:26.275398 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277d04f-7444-4182-9d7e-5f5410a3e431 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:26.278276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0277d04f-7444-4182-9d7e-5f5410a3e431 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:26.306714 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 812/1628] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:26.413587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3ac75-18fb-4f93-ad12-0b3d13d6c150 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:26.413803 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3ac75-18fb-4f93-ad12-0b3d13d6c150 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:26.414084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3ac75-18fb-4f93-ad12-0b3d13d6c150 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:26.415761 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 813/1629] 10.222.0.100 () {54 vars in 917 bytes} [Mon Aug 1 15:14:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:14:26.422046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8e726-8852-4605-8322-19191acb9e41 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:26.422354 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8e726-8852-4605-8322-19191acb9e41 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:26.422602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f8e726-8852-4605-8322-19191acb9e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:26.449478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 817/1630] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:26.452064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02d508-3d9e-4a21-8db3-2be7a67d2d92 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:26.462829 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b7f8e726-8852-4605-8322-19191acb9e41 None None] MFA Rules not processed for user `ec50ac7c54324884a1b2a6c63d617907`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:26.464172 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b7f8e726-8852-4605-8322-19191acb9e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:26.466848 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:26.530078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b02d508-3d9e-4a21-8db3-2be7a67d2d92 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:26.534578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da970406-e3c9-4451-afab-4b584dc6510c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:26.536476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da970406-e3c9-4451-afab-4b584dc6510c service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:26.537187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da970406-e3c9-4451-afab-4b584dc6510c service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:26.537647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da970406-e3c9-4451-afab-4b584dc6510c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:26.549911 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da970406-e3c9-4451-afab-4b584dc6510c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:26.560261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da970406-e3c9-4451-afab-4b584dc6510c service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:26.587597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 814/1631] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:14:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:26.588313 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 818/1632] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:26.603780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f8e726-8852-4605-8322-19191acb9e41 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:26.615079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f8e726-8852-4605-8322-19191acb9e41 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:26.619121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f47d7-50a5-4d75-b679-f5f032f51d93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:26.619878 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f47d7-50a5-4d75-b679-f5f032f51d93 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:26.620261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f47d7-50a5-4d75-b679-f5f032f51d93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:26.625734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f47d7-50a5-4d75-b679-f5f032f51d93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:26.650602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f47d7-50a5-4d75-b679-f5f032f51d93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:26.657301 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f47d7-50a5-4d75-b679-f5f032f51d93 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:26.685227 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 815/1633] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:26.828374 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bc59b-2789-45d8-ad2a-9e92ba6d7b34 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:26.828948 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bc59b-2789-45d8-ad2a-9e92ba6d7b34 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:26.829437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bc59b-2789-45d8-ad2a-9e92ba6d7b34 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:26.831407 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 819/1634] 10.222.0.100 () {54 vars in 917 bytes} [Mon Aug 1 15:14:26 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:14:26.852795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137253d2-72e0-49ea-88bf-afab21540e4f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:26.853116 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137253d2-72e0-49ea-88bf-afab21540e4f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:26.854309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bc59b-2789-45d8-ad2a-9e92ba6d7b34 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:26.856775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137253d2-72e0-49ea-88bf-afab21540e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:26.891430 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bc59b-2789-45d8-ad2a-9e92ba6d7b34 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:26.902699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d74f82-314c-46be-9947-607f8ef19c26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:26.903354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d74f82-314c-46be-9947-607f8ef19c26 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:26.903642 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d74f82-314c-46be-9947-607f8ef19c26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:26.903909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d74f82-314c-46be-9947-607f8ef19c26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:26.920874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-137253d2-72e0-49ea-88bf-afab21540e4f None None] MFA Rules not processed for user `ec50ac7c54324884a1b2a6c63d617907`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:26.937887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-137253d2-72e0-49ea-88bf-afab21540e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:26.991058 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 816/1635] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:14:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:27.093020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d74f82-314c-46be-9947-607f8ef19c26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:27.095295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d74f82-314c-46be-9947-607f8ef19c26 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:27.117317 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 820/1636] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:27.505316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137253d2-72e0-49ea-88bf-afab21540e4f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:27.518162 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137253d2-72e0-49ea-88bf-afab21540e4f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:27.529743 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50cb8f-3a80-4a23-8453-8fb51158b7a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:27.530786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50cb8f-3a80-4a23-8453-8fb51158b7a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:27.532915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50cb8f-3a80-4a23-8453-8fb51158b7a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:27.533501 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50cb8f-3a80-4a23-8453-8fb51158b7a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:27.571072 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50cb8f-3a80-4a23-8453-8fb51158b7a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:27.572698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50cb8f-3a80-4a23-8453-8fb51158b7a7 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:27.596807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 817/1637] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:28.104829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d74f82-314c-46be-9947-607f8ef19c26 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:28.249574 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d74f82-314c-46be-9947-607f8ef19c26 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:28.255042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed640352-c265-4e74-a2f1-85032840a2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:28.255826 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed640352-c265-4e74-a2f1-85032840a2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:28.256128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed640352-c265-4e74-a2f1-85032840a2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:28.256709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed640352-c265-4e74-a2f1-85032840a2a8 admin admin] PATH_INFO: `/v3/users/47f5aec582c742b18cb17bddd77f2891` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:28.258778 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed640352-c265-4e74-a2f1-85032840a2a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f5aec582c742b18cb17bddd77f2891)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:28.261941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed640352-c265-4e74-a2f1-85032840a2a8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:28.411487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ed640352-c265-4e74-a2f1-85032840a2a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f5aec582c742b18cb17bddd77f2891'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:28.412597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ed640352-c265-4e74-a2f1-85032840a2a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f5aec582c742b18cb17bddd77f2891'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:28.438893 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:28.440923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ed640352-c265-4e74-a2f1-85032840a2a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f5aec582c742b18cb17bddd77f2891'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:28.442778 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ed640352-c265-4e74-a2f1-85032840a2a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f5aec582c742b18cb17bddd77f2891'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:28.475867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ed640352-c265-4e74-a2f1-85032840a2a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f5aec582c742b18cb17bddd77f2891'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:28.503994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e97d3d-d297-4c16-9266-7eacd9e3548e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:28.504297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e97d3d-d297-4c16-9266-7eacd9e3548e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:28.504575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e97d3d-d297-4c16-9266-7eacd9e3548e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:28.507880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 821/1638] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:14:28 2022] DELETE /identity/v3/users/47f5aec582c742b18cb17bddd77f2891 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:28.576802 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f1e97d3d-d297-4c16-9266-7eacd9e3548e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:28.579186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f1e97d3d-d297-4c16-9266-7eacd9e3548e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:28.726630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 818/1639] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:28.751728 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86173f-3ba5-4c9b-abdd-de1c4a006b07 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:28.752250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86173f-3ba5-4c9b-abdd-de1c4a006b07 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:28.752968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86173f-3ba5-4c9b-abdd-de1c4a006b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:28.789508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e97d3d-d297-4c16-9266-7eacd9e3548e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:28.791349 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f1e97d3d-d297-4c16-9266-7eacd9e3548e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:28.821953 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8d86173f-3ba5-4c9b-abdd-de1c4a006b07 None None] MFA Rules not processed for user `ea431174273e48e5bdd7e2ab0a03f14f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:28.826982 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8d86173f-3ba5-4c9b-abdd-de1c4a006b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:28.972175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e97d3d-d297-4c16-9266-7eacd9e3548e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:28.988805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d127e7-df75-4cb8-8210-5a4d141410f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:28.992611 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d127e7-df75-4cb8-8210-5a4d141410f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:28.993622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d127e7-df75-4cb8-8210-5a4d141410f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:28.994781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d127e7-df75-4cb8-8210-5a4d141410f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:29.012733 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:29.023025 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d127e7-df75-4cb8-8210-5a4d141410f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:29.028265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d127e7-df75-4cb8-8210-5a4d141410f1 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:29.035249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 822/1640] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:28 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:29.040292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86173f-3ba5-4c9b-abdd-de1c4a006b07 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:29.041248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8d86173f-3ba5-4c9b-abdd-de1c4a006b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:29.068272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86173f-3ba5-4c9b-abdd-de1c4a006b07 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:29.073829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668dd45-30de-4b30-bfa5-e27aa9b4333e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:29.074468 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668dd45-30de-4b30-bfa5-e27aa9b4333e admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:29.074788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668dd45-30de-4b30-bfa5-e27aa9b4333e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:29.075074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668dd45-30de-4b30-bfa5-e27aa9b4333e admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:29.075829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668dd45-30de-4b30-bfa5-e27aa9b4333e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:29.077690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7668dd45-30de-4b30-bfa5-e27aa9b4333e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:29.078118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 819/1641] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:14:29.117787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 823/1642] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:29 2022] POST /identity/v3/projects => generated 406 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:14:29.121883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d127e7-df75-4cb8-8210-5a4d141410f1 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:29.134601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d127e7-df75-4cb8-8210-5a4d141410f1 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:29.254081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108bea0-3595-4ad9-b848-7f7f85dd7e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:29.254081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108bea0-3595-4ad9-b848-7f7f85dd7e08 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:29.254081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108bea0-3595-4ad9-b848-7f7f85dd7e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:29.254081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108bea0-3595-4ad9-b848-7f7f85dd7e08 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:29.254827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108bea0-3595-4ad9-b848-7f7f85dd7e08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:29.257309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108bea0-3595-4ad9-b848-7f7f85dd7e08 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:29.285356 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 820/1643] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:29 2022] POST /identity/v3/users => generated 310 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:29.288505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668dd45-30de-4b30-bfa5-e27aa9b4333e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:29.301509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668dd45-30de-4b30-bfa5-e27aa9b4333e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:29.307724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283c0db-95a9-4188-85db-e47887cd820d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:29.308620 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283c0db-95a9-4188-85db-e47887cd820d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:29.308906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283c0db-95a9-4188-85db-e47887cd820d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:29.309354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283c0db-95a9-4188-85db-e47887cd820d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:29.310080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283c0db-95a9-4188-85db-e47887cd820d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:29.313734 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283c0db-95a9-4188-85db-e47887cd820d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:29.331048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 824/1644] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:29 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:29.337243 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108bea0-3595-4ad9-b848-7f7f85dd7e08 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:29.351425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108bea0-3595-4ad9-b848-7f7f85dd7e08 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:29.358494 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba727a-a437-4219-bc2c-637261903bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:29.359236 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba727a-a437-4219-bc2c-637261903bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:29.359671 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba727a-a437-4219-bc2c-637261903bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:29.359833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba727a-a437-4219-bc2c-637261903bc8 admin admin] PATH_INFO: `/v3/projects/7415bde6e4724d6aa960dc5bc35d3a69/users/8e0f46453e7948b180661239241d78f0/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:29.378870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba727a-a437-4219-bc2c-637261903bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7415bde6e4724d6aa960dc5bc35d3a69, user_id=8e0f46453e7948b180661239241d78f0, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:29.381989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ba727a-a437-4219-bc2c-637261903bc8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:29.402837 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 821/1645] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:29 2022] PUT /identity/v3/projects/7415bde6e4724d6aa960dc5bc35d3a69/users/8e0f46453e7948b180661239241d78f0/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:29.406859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283c0db-95a9-4188-85db-e47887cd820d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:29.420274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283c0db-95a9-4188-85db-e47887cd820d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:29.524765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8c16a-50f8-4a31-8e08-af6b912ea61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:29.526437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8c16a-50f8-4a31-8e08-af6b912ea61c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:29.527654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8c16a-50f8-4a31-8e08-af6b912ea61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:29.528354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8c16a-50f8-4a31-8e08-af6b912ea61c admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:29.529538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8c16a-50f8-4a31-8e08-af6b912ea61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:29.532667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8c16a-50f8-4a31-8e08-af6b912ea61c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:29.556881 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 825/1646] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:29 2022] GET /identity/v3/roles => generated 1427 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:29.561459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba727a-a437-4219-bc2c-637261903bc8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:29.578947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba727a-a437-4219-bc2c-637261903bc8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:29.584746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439f1c1-65b1-4265-9b52-1a26cbbe41d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:29.585471 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439f1c1-65b1-4265-9b52-1a26cbbe41d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:29.585756 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439f1c1-65b1-4265-9b52-1a26cbbe41d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:29.586039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439f1c1-65b1-4265-9b52-1a26cbbe41d6 admin admin] PATH_INFO: `/v3/projects/7415bde6e4724d6aa960dc5bc35d3a69/users/8e0f46453e7948b180661239241d78f0/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:29.588381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439f1c1-65b1-4265-9b52-1a26cbbe41d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7415bde6e4724d6aa960dc5bc35d3a69, user_id=8e0f46453e7948b180661239241d78f0, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:29.593344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439f1c1-65b1-4265-9b52-1a26cbbe41d6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:29.608746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 822/1647] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:29 2022] PUT /identity/v3/projects/7415bde6e4724d6aa960dc5bc35d3a69/users/8e0f46453e7948b180661239241d78f0/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:29.746711 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8c16a-50f8-4a31-8e08-af6b912ea61c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:29.749020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-72a8c16a-50f8-4a31-8e08-af6b912ea61c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:29.766457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8c16a-50f8-4a31-8e08-af6b912ea61c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:29.790915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:29.874644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:29.875554 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:29.875848 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:29.876128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6 admin admin] PATH_INFO: `/v3/projects/b7d2693930994c6c990461c488626517` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:29.877418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d2693930994c6c990461c488626517)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:29.878472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: WARNING py.warnings [None req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:14:29.878472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: warnings.warn(msg) Aug 01 15:14:29.878472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: Aug 01 15:14:29.879792 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:29.963011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d2693930994c6c990461c488626517', 'request_id': 'req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:29.984561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d2693930994c6c990461c488626517', 'request_id': 'req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:29.995259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d2693930994c6c990461c488626517', 'request_id': 'req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.022640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6 admin admin] The token cache is being invalidate because project b7d2693930994c6c990461c488626517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:30.023035 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.024073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e6a10ade-4d01-4fd1-a2e3-28b3f2e038d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.030401 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 826/1648] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:14:29 2022] DELETE /identity/v3/projects/b7d2693930994c6c990461c488626517 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:30.031986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7537b7-5e21-4762-bb92-702ff5a4cfa7 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:30.032551 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7537b7-5e21-4762-bb92-702ff5a4cfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:30.032551 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7537b7-5e21-4762-bb92-702ff5a4cfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:30.065647 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3d7537b7-5e21-4762-bb92-702ff5a4cfa7 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:30.066973 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3d7537b7-5e21-4762-bb92-702ff5a4cfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:30.296132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 823/1649] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:30.303477 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf474f-e4e1-4598-8bf6-e9f7cff8f412 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:30.304831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf474f-e4e1-4598-8bf6-e9f7cff8f412 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:30.304831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf474f-e4e1-4598-8bf6-e9f7cff8f412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:30.361732 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-72cf474f-e4e1-4598-8bf6-e9f7cff8f412 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:30.363034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-72cf474f-e4e1-4598-8bf6-e9f7cff8f412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:30.383755 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 827/1650] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:30.389136 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7537b7-5e21-4762-bb92-702ff5a4cfa7 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:30.404175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7537b7-5e21-4762-bb92-702ff5a4cfa7 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:30.421664 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009693d-a12a-4d35-9ab9-2ac9e05d4e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:30.422309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009693d-a12a-4d35-9ab9-2ac9e05d4e86 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:30.422594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009693d-a12a-4d35-9ab9-2ac9e05d4e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:30.422907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0009693d-a12a-4d35-9ab9-2ac9e05d4e86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:30.423645 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009693d-a12a-4d35-9ab9-2ac9e05d4e86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:30.423940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009693d-a12a-4d35-9ab9-2ac9e05d4e86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:30.425573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0009693d-a12a-4d35-9ab9-2ac9e05d4e86 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:30.462779 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 824/1651] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:30.467272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf474f-e4e1-4598-8bf6-e9f7cff8f412 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:30.481762 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf474f-e4e1-4598-8bf6-e9f7cff8f412 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:30.492546 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7535e-654d-4fca-a5a3-6327593da6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:30.493241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7535e-654d-4fca-a5a3-6327593da6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:30.493510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7535e-654d-4fca-a5a3-6327593da6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:30.493858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7535e-654d-4fca-a5a3-6327593da6b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:30.494569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7535e-654d-4fca-a5a3-6327593da6b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:30.503071 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7535e-654d-4fca-a5a3-6327593da6b0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:30.545325 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009693d-a12a-4d35-9ab9-2ac9e05d4e86 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:30.547639 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0009693d-a12a-4d35-9ab9-2ac9e05d4e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:30.567349 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0009693d-a12a-4d35-9ab9-2ac9e05d4e86 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:30.577729 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 828/1652] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:30 2022] POST /identity/v3/projects => generated 406 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:30.579134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7535e-654d-4fca-a5a3-6327593da6b0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:30.586278 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b077fd-36de-4857-8b3b-fc3f54dca1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:30.587610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b077fd-36de-4857-8b3b-fc3f54dca1e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:30.588091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b077fd-36de-4857-8b3b-fc3f54dca1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:30.588532 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b077fd-36de-4857-8b3b-fc3f54dca1e0 admin admin] PATH_INFO: `/v3/users/7f811f71df43405b81e27cfc6045f1cf` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:30.590174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b077fd-36de-4857-8b3b-fc3f54dca1e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f811f71df43405b81e27cfc6045f1cf)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:30.591390 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: WARNING py.warnings [None req-88b077fd-36de-4857-8b3b-fc3f54dca1e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 15:14:30.591390 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: warnings.warn(msg) Aug 01 15:14:30.591390 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: Aug 01 15:14:30.593109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b077fd-36de-4857-8b3b-fc3f54dca1e0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:30.672667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7535e-654d-4fca-a5a3-6327593da6b0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:30.677828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721e523-7eec-48f4-80da-9700547272ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:30.678907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721e523-7eec-48f4-80da-9700547272ff admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:30.679294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721e523-7eec-48f4-80da-9700547272ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:30.679639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721e523-7eec-48f4-80da-9700547272ff admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:30.680700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721e523-7eec-48f4-80da-9700547272ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:30.684105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721e523-7eec-48f4-80da-9700547272ff admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:30.700249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:30.768320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 829/1653] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:30 2022] POST /identity/v3/users => generated 304 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:30.769302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721e523-7eec-48f4-80da-9700547272ff admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:30.778531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-88b077fd-36de-4857-8b3b-fc3f54dca1e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f811f71df43405b81e27cfc6045f1cf'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.788385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721e523-7eec-48f4-80da-9700547272ff admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:30.797082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-88b077fd-36de-4857-8b3b-fc3f54dca1e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f811f71df43405b81e27cfc6045f1cf'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.799359 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-88b077fd-36de-4857-8b3b-fc3f54dca1e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f811f71df43405b81e27cfc6045f1cf'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.800111 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-88b077fd-36de-4857-8b3b-fc3f54dca1e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f811f71df43405b81e27cfc6045f1cf'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.800422 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a66c1-9e7a-4c4a-abc3-a82efc846fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:30.801075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a66c1-9e7a-4c4a-abc3-a82efc846fbb admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:30.801354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a66c1-9e7a-4c4a-abc3-a82efc846fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:30.801648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a66c1-9e7a-4c4a-abc3-a82efc846fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:30.802351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a66c1-9e7a-4c4a-abc3-a82efc846fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:30.804022 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a66c1-9e7a-4c4a-abc3-a82efc846fbb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:30.836171 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 830/1654] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:30 2022] GET /identity/v3/roles => generated 1427 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:30.837160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a66c1-9e7a-4c4a-abc3-a82efc846fbb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:30.840835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2c6a66c1-9e7a-4c4a-abc3-a82efc846fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:30.861884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:30.874293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a66c1-9e7a-4c4a-abc3-a82efc846fbb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:30.877434 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-88b077fd-36de-4857-8b3b-fc3f54dca1e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f811f71df43405b81e27cfc6045f1cf'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:30.887191 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a70a9-6d3e-411f-a954-876bcdd18dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:30.888033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a70a9-6d3e-411f-a954-876bcdd18dbc admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:30.888575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a70a9-6d3e-411f-a954-876bcdd18dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:30.888855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a70a9-6d3e-411f-a954-876bcdd18dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:30.889570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a70a9-6d3e-411f-a954-876bcdd18dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:30.891217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a70a9-6d3e-411f-a954-876bcdd18dbc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:30.935159 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 831/1655] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:30 2022] GET /identity/v3/roles => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:30.937451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a70a9-6d3e-411f-a954-876bcdd18dbc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:30.940115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 825/1656] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:14:30 2022] DELETE /identity/v3/users/7f811f71df43405b81e27cfc6045f1cf => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:30.948496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbe009-6bb8-4aeb-9e6f-b65284803aaf None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:30.949043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbe009-6bb8-4aeb-9e6f-b65284803aaf None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:30.949584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbe009-6bb8-4aeb-9e6f-b65284803aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:30.958012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a70a9-6d3e-411f-a954-876bcdd18dbc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:30.997960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7acbe009-6bb8-4aeb-9e6f-b65284803aaf None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:31.000454 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7acbe009-6bb8-4aeb-9e6f-b65284803aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:31.090448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ca9f9-ae49-4d86-9595-22afed3656aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:31.091212 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ca9f9-ae49-4d86-9595-22afed3656aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:31.091550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ca9f9-ae49-4d86-9595-22afed3656aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:31.091827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ca9f9-ae49-4d86-9595-22afed3656aa admin admin] PATH_INFO: `/v3/projects/dfb429229c3745b6ad5fe73daa891559/users/e919b80fcc7d4fceb91d6f0f2dc5ede2/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:31.139492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ca9f9-ae49-4d86-9595-22afed3656aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfb429229c3745b6ad5fe73daa891559, user_id=e919b80fcc7d4fceb91d6f0f2dc5ede2, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:31.142479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ca9f9-ae49-4d86-9595-22afed3656aa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:31.143663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 826/1657] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:31.181345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 832/1658] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:30 2022] PUT /identity/v3/projects/dfb429229c3745b6ad5fe73daa891559/users/e919b80fcc7d4fceb91d6f0f2dc5ede2/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:31.193291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e28c63-ed7a-4a63-86f1-95babddab92d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:31.193853 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e28c63-ed7a-4a63-86f1-95babddab92d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:31.194285 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e28c63-ed7a-4a63-86f1-95babddab92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:31.219690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ca9f9-ae49-4d86-9595-22afed3656aa admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:31.221828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ba1ca9f9-ae49-4d86-9595-22afed3656aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:31.271386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d1e28c63-ed7a-4a63-86f1-95babddab92d None None] MFA Rules not processed for user `e919b80fcc7d4fceb91d6f0f2dc5ede2`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:31.272695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d1e28c63-ed7a-4a63-86f1-95babddab92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:31.319549 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ca9f9-ae49-4d86-9595-22afed3656aa admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:31.324142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7233a-4e69-49ef-962e-b8bc4c2b36ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:31.324885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7233a-4e69-49ef-962e-b8bc4c2b36ce service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:31.326635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7233a-4e69-49ef-962e-b8bc4c2b36ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:31.327341 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7233a-4e69-49ef-962e-b8bc4c2b36ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:31.438605 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7233a-4e69-49ef-962e-b8bc4c2b36ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:31.440998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b7233a-4e69-49ef-962e-b8bc4c2b36ce service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:31.455360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 833/1659] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:31.456181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7233a-4e69-49ef-962e-b8bc4c2b36ce service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:31.459039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 827/1660] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:31 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:31.468948 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7233a-4e69-49ef-962e-b8bc4c2b36ce service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:31.474912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9e7c6-3084-4614-a12b-9cba89c199c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:31.475558 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9e7c6-3084-4614-a12b-9cba89c199c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:31.476409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9e7c6-3084-4614-a12b-9cba89c199c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:31.476702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9e7c6-3084-4614-a12b-9cba89c199c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:31.477486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9e7c6-3084-4614-a12b-9cba89c199c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:31.480943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9e7c6-3084-4614-a12b-9cba89c199c3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:31.520847 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 834/1661] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:31 2022] POST /identity/v3/projects => generated 404 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:31.524545 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e28c63-ed7a-4a63-86f1-95babddab92d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:31.541607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e28c63-ed7a-4a63-86f1-95babddab92d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:31.616268 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:31.681165 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21a5c2-4798-40a0-94c7-b01e9ef6db3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:31.682514 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21a5c2-4798-40a0-94c7-b01e9ef6db3e admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:31.682952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21a5c2-4798-40a0-94c7-b01e9ef6db3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:31.683236 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21a5c2-4798-40a0-94c7-b01e9ef6db3e admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:31.684788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21a5c2-4798-40a0-94c7-b01e9ef6db3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:31.687604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21a5c2-4798-40a0-94c7-b01e9ef6db3e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:31.732458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 828/1662] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:31 2022] POST /identity/v3/users => generated 309 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:31.736298 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9e7c6-3084-4614-a12b-9cba89c199c3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:31.743437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:31.751771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9e7c6-3084-4614-a12b-9cba89c199c3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:31.758501 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a62369-dc84-4d7b-bed1-51a6e73e2b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:31.759372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a62369-dc84-4d7b-bed1-51a6e73e2b76 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:31.759659 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a62369-dc84-4d7b-bed1-51a6e73e2b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:31.759940 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a62369-dc84-4d7b-bed1-51a6e73e2b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:31.760713 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a62369-dc84-4d7b-bed1-51a6e73e2b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:31.763259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a62369-dc84-4d7b-bed1-51a6e73e2b76 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:31.780091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 835/1663] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:31 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:31.784262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21a5c2-4798-40a0-94c7-b01e9ef6db3e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:31.798892 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21a5c2-4798-40a0-94c7-b01e9ef6db3e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:31.805102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420b0b4-4fd7-4050-8c80-d0b0906fdbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:31.805812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420b0b4-4fd7-4050-8c80-d0b0906fdbe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:31.806097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420b0b4-4fd7-4050-8c80-d0b0906fdbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:31.806396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b420b0b4-4fd7-4050-8c80-d0b0906fdbe7 admin admin] PATH_INFO: `/v3/projects/32ba495a23874c6a87e65aa223612996/users/821f556bd48d465a8f7945c8b70348d0/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:31.856537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420b0b4-4fd7-4050-8c80-d0b0906fdbe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ba495a23874c6a87e65aa223612996, user_id=821f556bd48d465a8f7945c8b70348d0, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:31.858748 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b420b0b4-4fd7-4050-8c80-d0b0906fdbe7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:31.881936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 829/1664] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:31 2022] PUT /identity/v3/projects/32ba495a23874c6a87e65aa223612996/users/821f556bd48d465a8f7945c8b70348d0/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:14:31.883665 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a62369-dc84-4d7b-bed1-51a6e73e2b76 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:31.918586 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a62369-dc84-4d7b-bed1-51a6e73e2b76 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:31.964780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420b0b4-4fd7-4050-8c80-d0b0906fdbe7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:31.967314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b420b0b4-4fd7-4050-8c80-d0b0906fdbe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:32.090577 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7230b0-cf51-48bf-b050-10ffd397ca0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:32.091479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7230b0-cf51-48bf-b050-10ffd397ca0d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:32.091912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7230b0-cf51-48bf-b050-10ffd397ca0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:32.092329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7230b0-cf51-48bf-b050-10ffd397ca0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:32.093201 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7230b0-cf51-48bf-b050-10ffd397ca0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:32.096019 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7230b0-cf51-48bf-b050-10ffd397ca0d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:32.149999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 836/1665] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:31 2022] GET /identity/v3/roles => generated 1427 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:32.155382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7230b0-cf51-48bf-b050-10ffd397ca0d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:32.162062 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b420b0b4-4fd7-4050-8c80-d0b0906fdbe7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:32.187146 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7230b0-cf51-48bf-b050-10ffd397ca0d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:32.187349 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd44c26-fbeb-4ffe-9a50-de027df692ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:32.188048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd44c26-fbeb-4ffe-9a50-de027df692ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:32.188330 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd44c26-fbeb-4ffe-9a50-de027df692ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:32.190132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd44c26-fbeb-4ffe-9a50-de027df692ff admin admin] PATH_INFO: `/v3/users/aed8ce0800f8449fb1319bcb7ee3d9e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:32.195960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd44c26-fbeb-4ffe-9a50-de027df692ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=aed8ce0800f8449fb1319bcb7ee3d9e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:32.198314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd44c26-fbeb-4ffe-9a50-de027df692ff admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:32.207994 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fe8fa-9711-4b8c-8cbd-912dbce44632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:32.208945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fe8fa-9711-4b8c-8cbd-912dbce44632 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:32.209263 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fe8fa-9711-4b8c-8cbd-912dbce44632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:32.209551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fe8fa-9711-4b8c-8cbd-912dbce44632 admin admin] PATH_INFO: `/v3/projects/32ba495a23874c6a87e65aa223612996/users/821f556bd48d465a8f7945c8b70348d0/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:32.227922 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fe8fa-9711-4b8c-8cbd-912dbce44632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ba495a23874c6a87e65aa223612996, user_id=821f556bd48d465a8f7945c8b70348d0, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:32.231261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fe8fa-9711-4b8c-8cbd-912dbce44632 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:32.268697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 837/1666] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:32 2022] PUT /identity/v3/projects/32ba495a23874c6a87e65aa223612996/users/821f556bd48d465a8f7945c8b70348d0/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:32.280180 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880c580-c378-42ce-ba8a-4180f72cab59 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:32.281048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880c580-c378-42ce-ba8a-4180f72cab59 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:32.281048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880c580-c378-42ce-ba8a-4180f72cab59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:32.360134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8bd44c26-fbeb-4ffe-9a50-de027df692ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aed8ce0800f8449fb1319bcb7ee3d9e5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:32.376795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8bd44c26-fbeb-4ffe-9a50-de027df692ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aed8ce0800f8449fb1319bcb7ee3d9e5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:32.383361 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8bd44c26-fbeb-4ffe-9a50-de027df692ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aed8ce0800f8449fb1319bcb7ee3d9e5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:32.384722 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8bd44c26-fbeb-4ffe-9a50-de027df692ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aed8ce0800f8449fb1319bcb7ee3d9e5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:32.394173 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d880c580-c378-42ce-ba8a-4180f72cab59 None None] MFA Rules not processed for user `821f556bd48d465a8f7945c8b70348d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:32.397114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d880c580-c378-42ce-ba8a-4180f72cab59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:32.417853 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8bd44c26-fbeb-4ffe-9a50-de027df692ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aed8ce0800f8449fb1319bcb7ee3d9e5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:32.460775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 830/1667] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:14:31 2022] DELETE /identity/v3/users/aed8ce0800f8449fb1319bcb7ee3d9e5 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:32.462644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580ff21-5354-4c5b-bab1-f0cbd59e807f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:32.463373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580ff21-5354-4c5b-bab1-f0cbd59e807f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:32.465156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580ff21-5354-4c5b-bab1-f0cbd59e807f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:32.511799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 838/1668] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:32 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:32.515141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e580ff21-5354-4c5b-bab1-f0cbd59e807f None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:32.516634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e580ff21-5354-4c5b-bab1-f0cbd59e807f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:32.525791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880c580-c378-42ce-ba8a-4180f72cab59 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:32.528066 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d880c580-c378-42ce-ba8a-4180f72cab59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:32.556071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:32.667773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 831/1669] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:32.675838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880c580-c378-42ce-ba8a-4180f72cab59 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:32.709415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:32.779027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580ff21-5354-4c5b-bab1-f0cbd59e807f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:32.780551 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e580ff21-5354-4c5b-bab1-f0cbd59e807f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:32.788469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa9605-0651-4ca1-9b2a-f5c276e55959 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:32.789276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa9605-0651-4ca1-9b2a-f5c276e55959 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:32.789565 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa9605-0651-4ca1-9b2a-f5c276e55959 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:32.789847 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa9605-0651-4ca1-9b2a-f5c276e55959 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:32.875667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa9605-0651-4ca1-9b2a-f5c276e55959 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:32.877683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa9605-0651-4ca1-9b2a-f5c276e55959 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:32.891794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 839/1670] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:32 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:32.980823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e580ff21-5354-4c5b-bab1-f0cbd59e807f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:32.985663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cdad88-0103-4b30-80dc-ef20ff00704e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:32.986311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdad88-0103-4b30-80dc-ef20ff00704e service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:32.986868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdad88-0103-4b30-80dc-ef20ff00704e service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:32.987144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdad88-0103-4b30-80dc-ef20ff00704e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:33.019828 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cdad88-0103-4b30-80dc-ef20ff00704e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:33.026676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cdad88-0103-4b30-80dc-ef20ff00704e service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:33.060705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 832/1671] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:14:33.162713 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa9605-0651-4ca1-9b2a-f5c276e55959 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:33.166517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-43aa9605-0651-4ca1-9b2a-f5c276e55959 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:33.200543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa9605-0651-4ca1-9b2a-f5c276e55959 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:33.209089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d160cba-5265-4f3c-ac6f-678e4cc7d141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:33.210198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d160cba-5265-4f3c-ac6f-678e4cc7d141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:33.210555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d160cba-5265-4f3c-ac6f-678e4cc7d141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:33.211333 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d160cba-5265-4f3c-ac6f-678e4cc7d141 admin admin] PATH_INFO: `/v3/users/e919b80fcc7d4fceb91d6f0f2dc5ede2` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:33.213578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d160cba-5265-4f3c-ac6f-678e4cc7d141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e919b80fcc7d4fceb91d6f0f2dc5ede2)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:33.217803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d160cba-5265-4f3c-ac6f-678e4cc7d141 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:33.383890 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d160cba-5265-4f3c-ac6f-678e4cc7d141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e919b80fcc7d4fceb91d6f0f2dc5ede2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:33.390258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d160cba-5265-4f3c-ac6f-678e4cc7d141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e919b80fcc7d4fceb91d6f0f2dc5ede2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:33.409463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d160cba-5265-4f3c-ac6f-678e4cc7d141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e919b80fcc7d4fceb91d6f0f2dc5ede2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:33.411331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d160cba-5265-4f3c-ac6f-678e4cc7d141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e919b80fcc7d4fceb91d6f0f2dc5ede2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:33.471837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0d160cba-5265-4f3c-ac6f-678e4cc7d141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e919b80fcc7d4fceb91d6f0f2dc5ede2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:33.522324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 840/1672] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:14:33 2022] DELETE /identity/v3/users/e919b80fcc7d4fceb91d6f0f2dc5ede2 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:33.527726 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2bd56-a6cc-4824-bef0-88fde9c4eec6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:33.528016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2bd56-a6cc-4824-bef0-88fde9c4eec6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:33.528251 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2bd56-a6cc-4824-bef0-88fde9c4eec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:33.567474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-63f2bd56-a6cc-4824-bef0-88fde9c4eec6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:33.569762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-63f2bd56-a6cc-4824-bef0-88fde9c4eec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:33.747315 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 833/1673] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:33.771950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d160cba-5265-4f3c-ac6f-678e4cc7d141 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:33.773899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0d160cba-5265-4f3c-ac6f-678e4cc7d141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:33.862585 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:33.908941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d160cba-5265-4f3c-ac6f-678e4cc7d141 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:33.917598 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e3dd-1cc9-4ec4-8daa-e361e6aa077d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:33.918282 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e3dd-1cc9-4ec4-8daa-e361e6aa077d service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:33.918539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e3dd-1cc9-4ec4-8daa-e361e6aa077d service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:33.923674 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279e3dd-1cc9-4ec4-8daa-e361e6aa077d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:33.956409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279e3dd-1cc9-4ec4-8daa-e361e6aa077d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:33.960353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279e3dd-1cc9-4ec4-8daa-e361e6aa077d service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:34.020834 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 841/1674] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:34.030526 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa29de2-1d4b-4cee-87fb-e1eaa68025d1 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:34.031694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa29de2-1d4b-4cee-87fb-e1eaa68025d1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:34.032620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa29de2-1d4b-4cee-87fb-e1eaa68025d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:34.073190 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e3dd-1cc9-4ec4-8daa-e361e6aa077d service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:34.076811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b279e3dd-1cc9-4ec4-8daa-e361e6aa077d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:34.108494 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-6aa29de2-1d4b-4cee-87fb-e1eaa68025d1 None None] MFA Rules not processed for user `8e0f46453e7948b180661239241d78f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:34.111006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6aa29de2-1d4b-4cee-87fb-e1eaa68025d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:34.128309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279e3dd-1cc9-4ec4-8daa-e361e6aa077d service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:34.138872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:34.140184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:34.140592 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:34.140895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] PATH_INFO: `/v3/projects/0e3cc1543ceb45bf97c8547a648f6e65` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:34.142422 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e3cc1543ceb45bf97c8547a648f6e65)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:34.147065 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:34.247644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:34.285227 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 834/1675] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:34 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:34.293110 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e3cc1543ceb45bf97c8547a648f6e65', 'request_id': 'req-3b929af3-0028-4e54-85c6-d409218d8081'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.306055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e3cc1543ceb45bf97c8547a648f6e65', 'request_id': 'req-3b929af3-0028-4e54-85c6-d409218d8081'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.318758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e3cc1543ceb45bf97c8547a648f6e65', 'request_id': 'req-3b929af3-0028-4e54-85c6-d409218d8081'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.367627 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] The token cache is being invalidate because project 0e3cc1543ceb45bf97c8547a648f6e65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:34.368527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.370990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.373501 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 842/1676] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:14:34 2022] DELETE /identity/v3/projects/0e3cc1543ceb45bf97c8547a648f6e65 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:34.377864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa29de2-1d4b-4cee-87fb-e1eaa68025d1 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:34.393886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa29de2-1d4b-4cee-87fb-e1eaa68025d1 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:34.548281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5923a-7100-4c8a-9604-c20e1623b681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:34.549116 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5923a-7100-4c8a-9604-c20e1623b681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:34.549285 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5923a-7100-4c8a-9604-c20e1623b681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:34.549518 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5923a-7100-4c8a-9604-c20e1623b681 admin admin] PATH_INFO: `/v3/users/dd113de2c2604932b9cee204da77e5f9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:34.553629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5923a-7100-4c8a-9604-c20e1623b681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd113de2c2604932b9cee204da77e5f9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:34.556430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced5923a-7100-4c8a-9604-c20e1623b681 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:34.623008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:34.626476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:34.653320 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ced5923a-7100-4c8a-9604-c20e1623b681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd113de2c2604932b9cee204da77e5f9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.654794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b929af3-0028-4e54-85c6-d409218d8081 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:34.669789 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee6235-b339-448a-8893-51149be70344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:34.672106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee6235-b339-448a-8893-51149be70344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:34.672409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ced5923a-7100-4c8a-9604-c20e1623b681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd113de2c2604932b9cee204da77e5f9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.672996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee6235-b339-448a-8893-51149be70344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:34.673406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ced5923a-7100-4c8a-9604-c20e1623b681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd113de2c2604932b9cee204da77e5f9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.673819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee6235-b339-448a-8893-51149be70344 admin admin] PATH_INFO: `/v3/projects/696478725b424726a5c81d6ea974f819` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:34.674272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ced5923a-7100-4c8a-9604-c20e1623b681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd113de2c2604932b9cee204da77e5f9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.680800 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee6235-b339-448a-8893-51149be70344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=696478725b424726a5c81d6ea974f819)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:34.689703 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee6235-b339-448a-8893-51149be70344 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:34.698800 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ced5923a-7100-4c8a-9604-c20e1623b681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd113de2c2604932b9cee204da77e5f9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.727650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 835/1677] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:14:34 2022] DELETE /identity/v3/users/dd113de2c2604932b9cee204da77e5f9 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:34.813322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-faee6235-b339-448a-8893-51149be70344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '696478725b424726a5c81d6ea974f819', 'request_id': 'req-faee6235-b339-448a-8893-51149be70344'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.828031 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-faee6235-b339-448a-8893-51149be70344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '696478725b424726a5c81d6ea974f819', 'request_id': 'req-faee6235-b339-448a-8893-51149be70344'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.844040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-faee6235-b339-448a-8893-51149be70344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '696478725b424726a5c81d6ea974f819', 'request_id': 'req-faee6235-b339-448a-8893-51149be70344'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.865731 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-faee6235-b339-448a-8893-51149be70344 admin admin] The token cache is being invalidate because project 696478725b424726a5c81d6ea974f819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:34.866025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-faee6235-b339-448a-8893-51149be70344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.867397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-faee6235-b339-448a-8893-51149be70344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:34.869784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 843/1678] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:14:34 2022] DELETE /identity/v3/projects/696478725b424726a5c81d6ea974f819 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:34.874801 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12809ba0-e4e0-4c4b-b641-96c07bfb1f28 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:34.875345 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12809ba0-e4e0-4c4b-b641-96c07bfb1f28 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:34.876007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12809ba0-e4e0-4c4b-b641-96c07bfb1f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:34.921485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-12809ba0-e4e0-4c4b-b641-96c07bfb1f28 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:34.922788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-12809ba0-e4e0-4c4b-b641-96c07bfb1f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:35.028673 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 836/1679] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:35.037195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47885f6b-686d-42c1-98a7-3b809b2544e2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:35.037495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47885f6b-686d-42c1-98a7-3b809b2544e2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:35.037769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47885f6b-686d-42c1-98a7-3b809b2544e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:35.075713 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-47885f6b-686d-42c1-98a7-3b809b2544e2 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:35.077108 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-47885f6b-686d-42c1-98a7-3b809b2544e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:35.097969 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 844/1680] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:35.102483 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12809ba0-e4e0-4c4b-b641-96c07bfb1f28 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:35.117524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12809ba0-e4e0-4c4b-b641-96c07bfb1f28 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:35.130147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bc0e2-6a55-4e4c-b12a-515626d7d56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:35.130147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bc0e2-6a55-4e4c-b12a-515626d7d56a admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:35.130438 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bc0e2-6a55-4e4c-b12a-515626d7d56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:35.130701 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bc0e2-6a55-4e4c-b12a-515626d7d56a admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:35.131420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bc0e2-6a55-4e4c-b12a-515626d7d56a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:35.131733 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bc0e2-6a55-4e4c-b12a-515626d7d56a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:35.133332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bc0e2-6a55-4e4c-b12a-515626d7d56a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:35.145752 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:35.170952 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47885f6b-686d-42c1-98a7-3b809b2544e2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:35.175953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 837/1681] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:35.181262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47885f6b-686d-42c1-98a7-3b809b2544e2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:35.193228 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7f36a-d01d-47f8-b9c1-f6797cf6872c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:35.194028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7f36a-d01d-47f8-b9c1-f6797cf6872c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:35.194311 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7f36a-d01d-47f8-b9c1-f6797cf6872c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:35.194582 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7f36a-d01d-47f8-b9c1-f6797cf6872c admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:35.195377 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7f36a-d01d-47f8-b9c1-f6797cf6872c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:35.199607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7f36a-d01d-47f8-b9c1-f6797cf6872c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:35.272451 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bc0e2-6a55-4e4c-b12a-515626d7d56a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:35.278696 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 845/1682] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:35 2022] POST /identity/v3/projects => generated 374 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:14:35.289559 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bc0e2-6a55-4e4c-b12a-515626d7d56a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:35.464942 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d425646-1e89-4f72-8305-912fe73cc75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:35.465592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d425646-1e89-4f72-8305-912fe73cc75b admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:35.465876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d425646-1e89-4f72-8305-912fe73cc75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:35.466156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d425646-1e89-4f72-8305-912fe73cc75b admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:35.466965 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d425646-1e89-4f72-8305-912fe73cc75b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:35.473561 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d425646-1e89-4f72-8305-912fe73cc75b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:35.559125 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7f36a-d01d-47f8-b9c1-f6797cf6872c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:35.561996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 838/1683] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:35 2022] POST /identity/v3/users => generated 288 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:35.574408 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7f36a-d01d-47f8-b9c1-f6797cf6872c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:35.588656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03343cb1-6d01-461a-a167-2e71b97d7b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:35.589918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03343cb1-6d01-461a-a167-2e71b97d7b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:35.590196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03343cb1-6d01-461a-a167-2e71b97d7b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:35.590552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03343cb1-6d01-461a-a167-2e71b97d7b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:35.591324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03343cb1-6d01-461a-a167-2e71b97d7b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:35.599617 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03343cb1-6d01-461a-a167-2e71b97d7b8c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:35.626212 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 846/1684] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:35 2022] GET /identity/v3/roles => generated 1427 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:35.630530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d425646-1e89-4f72-8305-912fe73cc75b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:35.642183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d425646-1e89-4f72-8305-912fe73cc75b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:35.662210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634405ad-8eaf-47a2-a983-907fdb776a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:35.663206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634405ad-8eaf-47a2-a983-907fdb776a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:35.663495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634405ad-8eaf-47a2-a983-907fdb776a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:35.663772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634405ad-8eaf-47a2-a983-907fdb776a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:35.664483 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634405ad-8eaf-47a2-a983-907fdb776a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:35.666182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634405ad-8eaf-47a2-a983-907fdb776a39 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:35.696827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 839/1685] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:35 2022] GET /identity/v3/roles => generated 1427 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:35.701487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03343cb1-6d01-461a-a167-2e71b97d7b8c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:35.717646 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03343cb1-6d01-461a-a167-2e71b97d7b8c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:35.729079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481828ad-942c-4852-a1c0-be04566295e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:35.729314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481828ad-942c-4852-a1c0-be04566295e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:35.729597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481828ad-942c-4852-a1c0-be04566295e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:35.732614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481828ad-942c-4852-a1c0-be04566295e5 admin admin] PATH_INFO: `/v3/projects/8fd938728e0144f39e0ca66e08b0738d/users/dbdaeedf7e494af7bafb1004fc6f23dc/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:35.738264 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:35.772811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481828ad-942c-4852-a1c0-be04566295e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd938728e0144f39e0ca66e08b0738d, user_id=dbdaeedf7e494af7bafb1004fc6f23dc, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:35.778212 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481828ad-942c-4852-a1c0-be04566295e5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:35.806723 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 847/1686] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:35 2022] PUT /identity/v3/projects/8fd938728e0144f39e0ca66e08b0738d/users/dbdaeedf7e494af7bafb1004fc6f23dc/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:35.820954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffd03a-b168-4a28-a6f3-7429d91d1769 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:35.821632 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffd03a-b168-4a28-a6f3-7429d91d1769 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:35.822107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daffd03a-b168-4a28-a6f3-7429d91d1769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:35.888397 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-daffd03a-b168-4a28-a6f3-7429d91d1769 None None] MFA Rules not processed for user `dbdaeedf7e494af7bafb1004fc6f23dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:35.890565 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-daffd03a-b168-4a28-a6f3-7429d91d1769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:36.062434 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 840/1687] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:36.067535 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481828ad-942c-4852-a1c0-be04566295e5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:36.097053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffd03a-b168-4a28-a6f3-7429d91d1769 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:36.099552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-daffd03a-b168-4a28-a6f3-7429d91d1769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:36.235412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481828ad-942c-4852-a1c0-be04566295e5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:36.239522 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daffd03a-b168-4a28-a6f3-7429d91d1769 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:36.245735 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d0958-646b-4f58-8bb9-a574c71e96ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:36.246463 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d0958-646b-4f58-8bb9-a574c71e96ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.246793 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d0958-646b-4f58-8bb9-a574c71e96ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.247229 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d0958-646b-4f58-8bb9-a574c71e96ea admin admin] PATH_INFO: `/v3/projects/743e5b018819415bbd5548d6df4351e1` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.248688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d0958-646b-4f58-8bb9-a574c71e96ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=743e5b018819415bbd5548d6df4351e1)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:36.251580 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d0958-646b-4f58-8bb9-a574c71e96ea admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:36.254754 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4ab91-1bc7-4f2c-8ef2-9895c81d41aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:36.255404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4ab91-1bc7-4f2c-8ef2-9895c81d41aa admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.255688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4ab91-1bc7-4f2c-8ef2-9895c81d41aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.255962 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4ab91-1bc7-4f2c-8ef2-9895c81d41aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.256760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4ab91-1bc7-4f2c-8ef2-9895c81d41aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:36.259045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4ab91-1bc7-4f2c-8ef2-9895c81d41aa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:36.259772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:36.333390 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-630d0958-646b-4f58-8bb9-a574c71e96ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '743e5b018819415bbd5548d6df4351e1', 'request_id': 'req-630d0958-646b-4f58-8bb9-a574c71e96ea'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:36.342510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 848/1688] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:36 2022] POST /identity/v3/projects => generated 374 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:36.343412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4ab91-1bc7-4f2c-8ef2-9895c81d41aa admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:36.350981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-630d0958-646b-4f58-8bb9-a574c71e96ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '743e5b018819415bbd5548d6df4351e1', 'request_id': 'req-630d0958-646b-4f58-8bb9-a574c71e96ea'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:36.367233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-630d0958-646b-4f58-8bb9-a574c71e96ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '743e5b018819415bbd5548d6df4351e1', 'request_id': 'req-630d0958-646b-4f58-8bb9-a574c71e96ea'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:36.386603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-630d0958-646b-4f58-8bb9-a574c71e96ea admin admin] The token cache is being invalidate because project 743e5b018819415bbd5548d6df4351e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:36.386994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-630d0958-646b-4f58-8bb9-a574c71e96ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:36.387608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-630d0958-646b-4f58-8bb9-a574c71e96ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:36.390875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 841/1689] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:14:36 2022] DELETE /identity/v3/projects/743e5b018819415bbd5548d6df4351e1 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:36.398086 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf217f7-f15e-444d-94b8-dae0d1ce7fb1 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.398388 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf217f7-f15e-444d-94b8-dae0d1ce7fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.398643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf217f7-f15e-444d-94b8-dae0d1ce7fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.432554 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9cf217f7-f15e-444d-94b8-dae0d1ce7fb1 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:36.439164 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9cf217f7-f15e-444d-94b8-dae0d1ce7fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:36.487260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4ab91-1bc7-4f2c-8ef2-9895c81d41aa admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:36.503749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39463a2f-c16a-4ec3-8eb5-a586b034dd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:36.504395 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39463a2f-c16a-4ec3-8eb5-a586b034dd3b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.504685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39463a2f-c16a-4ec3-8eb5-a586b034dd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.504959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39463a2f-c16a-4ec3-8eb5-a586b034dd3b admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.505639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39463a2f-c16a-4ec3-8eb5-a586b034dd3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:36.515932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39463a2f-c16a-4ec3-8eb5-a586b034dd3b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:36.560633 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 849/1690] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:36 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:36.566251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39463a2f-c16a-4ec3-8eb5-a586b034dd3b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:36.570066 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-39463a2f-c16a-4ec3-8eb5-a586b034dd3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:36.588041 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 842/1691] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:36.598288 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eef634-034b-4a4e-803c-0a76f53b859a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.598288 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eef634-034b-4a4e-803c-0a76f53b859a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.598547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eef634-034b-4a4e-803c-0a76f53b859a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.605554 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39463a2f-c16a-4ec3-8eb5-a586b034dd3b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:36.623237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781ec5c-8959-4d3c-b5e4-d82e3704efe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:36.623982 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781ec5c-8959-4d3c-b5e4-d82e3704efe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.624268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781ec5c-8959-4d3c-b5e4-d82e3704efe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.624543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781ec5c-8959-4d3c-b5e4-d82e3704efe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.625258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781ec5c-8959-4d3c-b5e4-d82e3704efe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:36.627333 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781ec5c-8959-4d3c-b5e4-d82e3704efe5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:36.644339 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-25eef634-034b-4a4e-803c-0a76f53b859a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:36.645584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-25eef634-034b-4a4e-803c-0a76f53b859a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:36.656641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 850/1692] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:36 2022] GET /identity/v3/roles => generated 1427 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:36.657603 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781ec5c-8959-4d3c-b5e4-d82e3704efe5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:36.676669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 843/1693] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:36.677803 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eef634-034b-4a4e-803c-0a76f53b859a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:36.683633 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781ec5c-8959-4d3c-b5e4-d82e3704efe5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:36.694030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eef634-034b-4a4e-803c-0a76f53b859a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:36.697362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbd422-17ee-4238-890d-19ff724e98f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:36.698283 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbd422-17ee-4238-890d-19ff724e98f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.698725 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbd422-17ee-4238-890d-19ff724e98f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.699140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbd422-17ee-4238-890d-19ff724e98f0 admin admin] PATH_INFO: `/v3/projects/8073ac3a235f478498080852b80ddbb3/users/9ca77de723554cb09468830ce73c3be4/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.711673 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514764eb-068c-4503-9b61-a8f9fabb0e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:36.712686 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514764eb-068c-4503-9b61-a8f9fabb0e9b admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.713573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514764eb-068c-4503-9b61-a8f9fabb0e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.714518 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514764eb-068c-4503-9b61-a8f9fabb0e9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.715387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514764eb-068c-4503-9b61-a8f9fabb0e9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:36.715848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514764eb-068c-4503-9b61-a8f9fabb0e9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:36.722023 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514764eb-068c-4503-9b61-a8f9fabb0e9b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:36.739217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbd422-17ee-4238-890d-19ff724e98f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8073ac3a235f478498080852b80ddbb3, user_id=9ca77de723554cb09468830ce73c3be4, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:36.742606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbd422-17ee-4238-890d-19ff724e98f0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:36.753447 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 844/1694] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:36.756458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514764eb-068c-4503-9b61-a8f9fabb0e9b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:36.774499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514764eb-068c-4503-9b61-a8f9fabb0e9b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:36.776251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 851/1695] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:36 2022] PUT /identity/v3/projects/8073ac3a235f478498080852b80ddbb3/users/9ca77de723554cb09468830ce73c3be4/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:36.780685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbd422-17ee-4238-890d-19ff724e98f0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:36.835292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:36.917413 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f229f19-a93c-4208-84f8-39bdea760c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:36.918056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f229f19-a93c-4208-84f8-39bdea760c0b admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.918344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f229f19-a93c-4208-84f8-39bdea760c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.918622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f229f19-a93c-4208-84f8-39bdea760c0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.919342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f229f19-a93c-4208-84f8-39bdea760c0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:36.921286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f229f19-a93c-4208-84f8-39bdea760c0b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:36.927192 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbd422-17ee-4238-890d-19ff724e98f0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:36.939044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd67554-6265-41c8-b346-b4a44549095e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:36.939932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd67554-6265-41c8-b346-b4a44549095e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:36.940353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd67554-6265-41c8-b346-b4a44549095e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:36.940768 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd67554-6265-41c8-b346-b4a44549095e admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:36.941683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd67554-6265-41c8-b346-b4a44549095e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:36.947493 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd67554-6265-41c8-b346-b4a44549095e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:36.968855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 852/1696] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:36 2022] GET /identity/v3/roles => generated 1427 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:36.976777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:36.981785 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd67554-6265-41c8-b346-b4a44549095e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:36.992994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 845/1697] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:36 2022] POST /identity/v3/projects => generated 384 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:36.996917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f229f19-a93c-4208-84f8-39bdea760c0b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:37.013158 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd67554-6265-41c8-b346-b4a44549095e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:37.118342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf1c06-728a-471c-b999-704a0dee0322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:37.119075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf1c06-728a-471c-b999-704a0dee0322 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:37.119362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf1c06-728a-471c-b999-704a0dee0322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:37.119643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf1c06-728a-471c-b999-704a0dee0322 admin admin] PATH_INFO: `/v3/projects/8073ac3a235f478498080852b80ddbb3/users/9ca77de723554cb09468830ce73c3be4/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:37.124159 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf1c06-728a-471c-b999-704a0dee0322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8073ac3a235f478498080852b80ddbb3, user_id=9ca77de723554cb09468830ce73c3be4, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:37.129372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf1c06-728a-471c-b999-704a0dee0322 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:37.159686 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f229f19-a93c-4208-84f8-39bdea760c0b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:37.160690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 853/1698] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:36 2022] PUT /identity/v3/projects/8073ac3a235f478498080852b80ddbb3/users/9ca77de723554cb09468830ce73c3be4/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:37.171992 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a99af4-fb61-4f6f-a478-9da4941cad02 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:37.172597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a99af4-fb61-4f6f-a478-9da4941cad02 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:37.172941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a99af4-fb61-4f6f-a478-9da4941cad02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:37.247995 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d7a99af4-fb61-4f6f-a478-9da4941cad02 None None] MFA Rules not processed for user `9ca77de723554cb09468830ce73c3be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:37.250841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d7a99af4-fb61-4f6f-a478-9da4941cad02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:37.329191 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528372d-2b9c-4d1c-bf36-33889b58e982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:37.331537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528372d-2b9c-4d1c-bf36-33889b58e982 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:37.331833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528372d-2b9c-4d1c-bf36-33889b58e982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:37.333239 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0528372d-2b9c-4d1c-bf36-33889b58e982 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:37.341188 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528372d-2b9c-4d1c-bf36-33889b58e982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:37.351027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0528372d-2b9c-4d1c-bf36-33889b58e982 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:37.420389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 846/1699] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:36 2022] POST /identity/v3/users => generated 293 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:37.427422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528372d-2b9c-4d1c-bf36-33889b58e982 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:37.448102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0528372d-2b9c-4d1c-bf36-33889b58e982 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:37.465392 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea697145-27de-44a2-b386-ae874e243078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:37.466218 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea697145-27de-44a2-b386-ae874e243078 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:37.466610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea697145-27de-44a2-b386-ae874e243078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:37.467055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea697145-27de-44a2-b386-ae874e243078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:37.467875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea697145-27de-44a2-b386-ae874e243078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:37.476320 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea697145-27de-44a2-b386-ae874e243078 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:37.484230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 854/1700] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:37.486424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a99af4-fb61-4f6f-a478-9da4941cad02 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:37.495654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d7a99af4-fb61-4f6f-a478-9da4941cad02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:37.525376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 847/1701] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:37 2022] GET /identity/v3/roles => generated 1427 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:37.527126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea697145-27de-44a2-b386-ae874e243078 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:37.544564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a99af4-fb61-4f6f-a478-9da4941cad02 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:37.548954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ea697145-27de-44a2-b386-ae874e243078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:37.571086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:37.571917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:37.572304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:37.572687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] PATH_INFO: `/v3/projects/dfb429229c3745b6ad5fe73daa891559` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:37.575382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfb429229c3745b6ad5fe73daa891559)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:37.579079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:37.692535 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:37.736445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea697145-27de-44a2-b386-ae874e243078 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:37.738023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfb429229c3745b6ad5fe73daa891559', 'request_id': 'req-75812d52-bf10-4c41-9156-c0f36083e7bc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:37.757477 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfb429229c3745b6ad5fe73daa891559', 'request_id': 'req-75812d52-bf10-4c41-9156-c0f36083e7bc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:37.772843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfb429229c3745b6ad5fe73daa891559', 'request_id': 'req-75812d52-bf10-4c41-9156-c0f36083e7bc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:37.799824 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] The token cache is being invalidate because project dfb429229c3745b6ad5fe73daa891559 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:37.800369 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:37.802451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:37.805037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 855/1702] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:14:37 2022] DELETE /identity/v3/projects/dfb429229c3745b6ad5fe73daa891559 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:37.806017 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:37.808129 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:37.849334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:37.877143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5065e-30b1-48e2-b2d7-20d09989c0e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:37.882717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5065e-30b1-48e2-b2d7-20d09989c0e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:37.883091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5065e-30b1-48e2-b2d7-20d09989c0e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:37.883481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5065e-30b1-48e2-b2d7-20d09989c0e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:37.886696 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a7b5065e-30b1-48e2-b2d7-20d09989c0e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:37.960154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75812d52-bf10-4c41-9156-c0f36083e7bc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:37.971596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24064307-690c-4cb9-862b-43311cf0fd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:37.974050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24064307-690c-4cb9-862b-43311cf0fd19 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:37.974446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24064307-690c-4cb9-862b-43311cf0fd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:37.974852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24064307-690c-4cb9-862b-43311cf0fd19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:37.976641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24064307-690c-4cb9-862b-43311cf0fd19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:37.982941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24064307-690c-4cb9-862b-43311cf0fd19 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:38.006772 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 856/1703] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:37 2022] GET /identity/v3/roles => generated 1427 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:38.013315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24064307-690c-4cb9-862b-43311cf0fd19 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:38.017484 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-24064307-690c-4cb9-862b-43311cf0fd19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:38.043519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24064307-690c-4cb9-862b-43311cf0fd19 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:38.045472 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5065e-30b1-48e2-b2d7-20d09989c0e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:38.048577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5065e-30b1-48e2-b2d7-20d09989c0e5 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:38.059052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3289d-0d77-4d6d-8c9e-204152050486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:38.060289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3289d-0d77-4d6d-8c9e-204152050486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:38.060759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3289d-0d77-4d6d-8c9e-204152050486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:38.061287 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3289d-0d77-4d6d-8c9e-204152050486 admin admin] PATH_INFO: `/v3/users/821f556bd48d465a8f7945c8b70348d0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:38.064653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3289d-0d77-4d6d-8c9e-204152050486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=821f556bd48d465a8f7945c8b70348d0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:38.067593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3289d-0d77-4d6d-8c9e-204152050486 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:38.075352 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 848/1704] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:38.077343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5065e-30b1-48e2-b2d7-20d09989c0e5 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:38.098517 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5065e-30b1-48e2-b2d7-20d09989c0e5 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:38.115291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b53e1-78ef-4f21-97ba-7f6e17dd8a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:38.115995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b53e1-78ef-4f21-97ba-7f6e17dd8a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:38.116507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b53e1-78ef-4f21-97ba-7f6e17dd8a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:38.116800 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b53e1-78ef-4f21-97ba-7f6e17dd8a68 admin admin] PATH_INFO: `/v3/projects/075ab2f308204d83b18d745778bbdef7/users/0944cc7d776f42fb9e11eea3cc193509/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:38.161307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b53e1-78ef-4f21-97ba-7f6e17dd8a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=075ab2f308204d83b18d745778bbdef7, user_id=0944cc7d776f42fb9e11eea3cc193509, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:38.169211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b53e1-78ef-4f21-97ba-7f6e17dd8a68 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:38.183784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36a3289d-0d77-4d6d-8c9e-204152050486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '821f556bd48d465a8f7945c8b70348d0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:38.184536 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36a3289d-0d77-4d6d-8c9e-204152050486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '821f556bd48d465a8f7945c8b70348d0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:38.198346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 849/1705] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:38 2022] PUT /identity/v3/projects/075ab2f308204d83b18d745778bbdef7/users/0944cc7d776f42fb9e11eea3cc193509/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:38.209582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72bb97-cfbd-4918-9826-d4e7161370ce None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:38.210049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72bb97-cfbd-4918-9826-d4e7161370ce None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:38.210439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72bb97-cfbd-4918-9826-d4e7161370ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:38.211264 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36a3289d-0d77-4d6d-8c9e-204152050486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '821f556bd48d465a8f7945c8b70348d0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:38.219324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36a3289d-0d77-4d6d-8c9e-204152050486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '821f556bd48d465a8f7945c8b70348d0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:38.248081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-36a3289d-0d77-4d6d-8c9e-204152050486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '821f556bd48d465a8f7945c8b70348d0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:38.269906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5a72bb97-cfbd-4918-9826-d4e7161370ce None None] MFA Rules not processed for user `0944cc7d776f42fb9e11eea3cc193509`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:38.271504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5a72bb97-cfbd-4918-9826-d4e7161370ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:38.272434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 857/1706] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:14:38 2022] DELETE /identity/v3/users/821f556bd48d465a8f7945c8b70348d0 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:38.273682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3289d-0d77-4d6d-8c9e-204152050486 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:38.275349 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-36a3289d-0d77-4d6d-8c9e-204152050486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:38.351234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:38.422010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3289d-0d77-4d6d-8c9e-204152050486 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:38.430076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ded3e-9a09-458d-bec5-7f837c253ed3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:38.430949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ded3e-9a09-458d-bec5-7f837c253ed3 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:38.431387 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ded3e-9a09-458d-bec5-7f837c253ed3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:38.431813 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ded3e-9a09-458d-bec5-7f837c253ed3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:38.444069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b89ded3e-9a09-458d-bec5-7f837c253ed3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:38.476507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 850/1707] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:38.477641 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72bb97-cfbd-4918-9826-d4e7161370ce None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:38.481850 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5a72bb97-cfbd-4918-9826-d4e7161370ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:38.624140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ded3e-9a09-458d-bec5-7f837c253ed3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:38.634328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ded3e-9a09-458d-bec5-7f837c253ed3 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:38.669707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 858/1708] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:38.800297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:38.813643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72bb97-cfbd-4918-9826-d4e7161370ce None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:38.851914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd39e95-e846-4045-9840-2d00bd9e145f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:38.852766 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd39e95-e846-4045-9840-2d00bd9e145f admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:38.853056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd39e95-e846-4045-9840-2d00bd9e145f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:38.853334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd39e95-e846-4045-9840-2d00bd9e145f admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:38.854028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd39e95-e846-4045-9840-2d00bd9e145f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:38.856642 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd39e95-e846-4045-9840-2d00bd9e145f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:38.939823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 851/1709] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:38 2022] POST /identity/v3/projects => generated 384 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:38.941533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ded3e-9a09-458d-bec5-7f837c253ed3 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:38.959331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ded3e-9a09-458d-bec5-7f837c253ed3 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:39.092856 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113b949-f181-41de-a7ec-cd42392da263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:39.095449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113b949-f181-41de-a7ec-cd42392da263 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:39.095911 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113b949-f181-41de-a7ec-cd42392da263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:39.096300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113b949-f181-41de-a7ec-cd42392da263 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:39.097366 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113b949-f181-41de-a7ec-cd42392da263 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:39.100752 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113b949-f181-41de-a7ec-cd42392da263 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:39.127698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896ed50-77b2-4494-a021-c924ade28f20 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:39.128234 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896ed50-77b2-4494-a021-c924ade28f20 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:39.128234 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896ed50-77b2-4494-a021-c924ade28f20 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:39.131763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 852/1710] 10.222.0.100 () {54 vars in 926 bytes} [Mon Aug 1 15:14:39 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:14:39.153888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 859/1711] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:38 2022] POST /identity/v3/users => generated 299 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:39.159307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896ed50-77b2-4494-a021-c924ade28f20 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:39.184426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896ed50-77b2-4494-a021-c924ade28f20 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:39.185519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a4411-3ce1-464d-925d-13697c9355ce None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:39.185809 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a4411-3ce1-464d-925d-13697c9355ce None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:39.186089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a4411-3ce1-464d-925d-13697c9355ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:39.209575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6db3ad-8eec-4234-ac40-bdcd08ff48ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:39.209729 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6db3ad-8eec-4234-ac40-bdcd08ff48ec admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:39.209947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6db3ad-8eec-4234-ac40-bdcd08ff48ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:39.210179 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6db3ad-8eec-4234-ac40-bdcd08ff48ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:39.213143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6db3ad-8eec-4234-ac40-bdcd08ff48ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:39.218408 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6db3ad-8eec-4234-ac40-bdcd08ff48ec admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:39.239427 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-b86a4411-3ce1-464d-925d-13697c9355ce None None] MFA Rules not processed for user `3bac5863e7bc4c7e854bd60912adb561`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:39.241089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b86a4411-3ce1-464d-925d-13697c9355ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:39.243712 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 853/1712] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:39 2022] GET /identity/v3/roles => generated 1427 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:39.248009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6db3ad-8eec-4234-ac40-bdcd08ff48ec admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:39.273946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6db3ad-8eec-4234-ac40-bdcd08ff48ec admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:39.285971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002770f7-ed5e-4c81-9a36-933532276e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:39.286606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002770f7-ed5e-4c81-9a36-933532276e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:39.286960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002770f7-ed5e-4c81-9a36-933532276e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:39.288681 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002770f7-ed5e-4c81-9a36-933532276e7b admin admin] PATH_INFO: `/v3/projects/02a4c24481f548cf81ea695c94121720/users/4684b684cb6c4c12818d1820b8995887/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:39.321349 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002770f7-ed5e-4c81-9a36-933532276e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=02a4c24481f548cf81ea695c94121720, user_id=4684b684cb6c4c12818d1820b8995887, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:39.330132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002770f7-ed5e-4c81-9a36-933532276e7b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:39.373849 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 854/1713] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:39 2022] PUT /identity/v3/projects/02a4c24481f548cf81ea695c94121720/users/4684b684cb6c4c12818d1820b8995887/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:39.378655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002770f7-ed5e-4c81-9a36-933532276e7b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:39.524884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 860/1714] 10.222.0.100 () {58 vars in 1023 bytes} [Mon Aug 1 15:14:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:39.562583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002770f7-ed5e-4c81-9a36-933532276e7b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:39.570028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab00498-8886-48fb-a80d-c4d2bcb01ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:39.570945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab00498-8886-48fb-a80d-c4d2bcb01ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:39.571774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab00498-8886-48fb-a80d-c4d2bcb01ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:39.572340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab00498-8886-48fb-a80d-c4d2bcb01ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:39.574622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab00498-8886-48fb-a80d-c4d2bcb01ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:39.582323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b145df0-0ea8-4afd-aa99-a4d76a6fd504 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:39.583008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b145df0-0ea8-4afd-aa99-a4d76a6fd504 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:39.583292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b145df0-0ea8-4afd-aa99-a4d76a6fd504 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:39.585023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 861/1715] 10.222.0.100 () {54 vars in 966 bytes} [Mon Aug 1 15:14:39 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:14:39.587685 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab00498-8886-48fb-a80d-c4d2bcb01ad3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:39.592424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c697a62-351c-48b3-8c01-7cedae5a369b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:39.592710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c697a62-351c-48b3-8c01-7cedae5a369b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:39.592992 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c697a62-351c-48b3-8c01-7cedae5a369b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:39.604905 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 855/1716] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:39 2022] GET /identity/v3/roles => generated 1427 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:39.610378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab00498-8886-48fb-a80d-c4d2bcb01ad3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:39.628049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab00498-8886-48fb-a80d-c4d2bcb01ad3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:39.657298 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0c697a62-351c-48b3-8c01-7cedae5a369b None None] MFA Rules not processed for user `3bac5863e7bc4c7e854bd60912adb561`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:39.658711 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0c697a62-351c-48b3-8c01-7cedae5a369b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:39.661626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793180c2-a35c-41c9-8fe7-4956eb61d669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:39.662581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793180c2-a35c-41c9-8fe7-4956eb61d669 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:39.662723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793180c2-a35c-41c9-8fe7-4956eb61d669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:39.662917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793180c2-a35c-41c9-8fe7-4956eb61d669 admin admin] PATH_INFO: `/v3/projects/02a4c24481f548cf81ea695c94121720/users/4684b684cb6c4c12818d1820b8995887/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:39.666885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793180c2-a35c-41c9-8fe7-4956eb61d669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02a4c24481f548cf81ea695c94121720, user_id=4684b684cb6c4c12818d1820b8995887, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:39.669497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793180c2-a35c-41c9-8fe7-4956eb61d669 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:39.675042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:39.713276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 856/1717] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:39 2022] PUT /identity/v3/projects/02a4c24481f548cf81ea695c94121720/users/4684b684cb6c4c12818d1820b8995887/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:39.715439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90ae8c-f04e-4615-b68d-c4e203071d40 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:39.715729 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90ae8c-f04e-4615-b68d-c4e203071d40 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:39.716009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c90ae8c-f04e-4615-b68d-c4e203071d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:39.775791 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0c90ae8c-f04e-4615-b68d-c4e203071d40 None None] MFA Rules not processed for user `4684b684cb6c4c12818d1820b8995887`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:39.777932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0c90ae8c-f04e-4615-b68d-c4e203071d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:39.840498 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 862/1718] 10.222.0.100 () {58 vars in 1063 bytes} [Mon Aug 1 15:14:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:39.848665 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c697a62-351c-48b3-8c01-7cedae5a369b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:39.866530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c697a62-351c-48b3-8c01-7cedae5a369b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:39.981990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 857/1719] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:39.990177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90ae8c-f04e-4615-b68d-c4e203071d40 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:40.023258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:40.046473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64a3eb-7b4e-4328-b0d1-e527635dcac9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:40.047334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64a3eb-7b4e-4328-b0d1-e527635dcac9 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:40.047619 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64a3eb-7b4e-4328-b0d1-e527635dcac9 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:40.047900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64a3eb-7b4e-4328-b0d1-e527635dcac9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:40.057850 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64a3eb-7b4e-4328-b0d1-e527635dcac9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:40.060162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64a3eb-7b4e-4328-b0d1-e527635dcac9 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:40.077038 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 863/1720] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:40.164773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c90ae8c-f04e-4615-b68d-c4e203071d40 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:40.175943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb055e-edc2-4eeb-94db-cda8d2e9b8e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:40.176622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb055e-edc2-4eeb-94db-cda8d2e9b8e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:40.176911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb055e-edc2-4eeb-94db-cda8d2e9b8e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:40.177174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb055e-edc2-4eeb-94db-cda8d2e9b8e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:40.196475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb055e-edc2-4eeb-94db-cda8d2e9b8e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:40.203876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb055e-edc2-4eeb-94db-cda8d2e9b8e7 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:40.231556 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 858/1721] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:14:40.451844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64a3eb-7b4e-4328-b0d1-e527635dcac9 service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:40.471402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64a3eb-7b4e-4328-b0d1-e527635dcac9 service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:40.484159 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ca06c-f16f-4105-8b81-02e55f5955ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:40.485536 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ca06c-f16f-4105-8b81-02e55f5955ab service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:40.486763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ca06c-f16f-4105-8b81-02e55f5955ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:40.487153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ca06c-f16f-4105-8b81-02e55f5955ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:40.636569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ca06c-f16f-4105-8b81-02e55f5955ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:40.638098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ca06c-f16f-4105-8b81-02e55f5955ab service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:40.639190 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb055e-edc2-4eeb-94db-cda8d2e9b8e7 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:40.640291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-cefb055e-edc2-4eeb-94db-cda8d2e9b8e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:40.667993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 864/1722] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:40.851160 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb055e-edc2-4eeb-94db-cda8d2e9b8e7 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:40.866351 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef9c98-82ef-493b-8682-30bab4d708a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:40.867101 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef9c98-82ef-493b-8682-30bab4d708a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:40.869462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef9c98-82ef-493b-8682-30bab4d708a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:40.869743 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef9c98-82ef-493b-8682-30bab4d708a8 admin admin] PATH_INFO: `/v3/projects/32ba495a23874c6a87e65aa223612996` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:40.873120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef9c98-82ef-493b-8682-30bab4d708a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32ba495a23874c6a87e65aa223612996)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:40.877402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef9c98-82ef-493b-8682-30bab4d708a8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:41.023620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3bef9c98-82ef-493b-8682-30bab4d708a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32ba495a23874c6a87e65aa223612996', 'request_id': 'req-3bef9c98-82ef-493b-8682-30bab4d708a8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:41.041656 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3bef9c98-82ef-493b-8682-30bab4d708a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32ba495a23874c6a87e65aa223612996', 'request_id': 'req-3bef9c98-82ef-493b-8682-30bab4d708a8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:41.061167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3bef9c98-82ef-493b-8682-30bab4d708a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32ba495a23874c6a87e65aa223612996', 'request_id': 'req-3bef9c98-82ef-493b-8682-30bab4d708a8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:41.084856 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3bef9c98-82ef-493b-8682-30bab4d708a8 admin admin] The token cache is being invalidate because project 32ba495a23874c6a87e65aa223612996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:41.085156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3bef9c98-82ef-493b-8682-30bab4d708a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:41.087834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3bef9c98-82ef-493b-8682-30bab4d708a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:41.096138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 859/1723] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:14:40 2022] DELETE /identity/v3/projects/32ba495a23874c6a87e65aa223612996 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:14:41.098413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fd764-9601-49f1-99b5-bdbb55ed6b1d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:41.098918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fd764-9601-49f1-99b5-bdbb55ed6b1d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:41.098988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fd764-9601-49f1-99b5-bdbb55ed6b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:41.155250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-124fd764-9601-49f1-99b5-bdbb55ed6b1d None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:41.157154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-124fd764-9601-49f1-99b5-bdbb55ed6b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:41.245777 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:41.326839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 865/1724] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:41.333252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ea78f-d4a7-425a-bca6-73cacde562a5 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:41.333730 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ea78f-d4a7-425a-bca6-73cacde562a5 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:41.333819 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ea78f-d4a7-425a-bca6-73cacde562a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:41.371630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9f2ea78f-d4a7-425a-bca6-73cacde562a5 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:41.372875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9f2ea78f-d4a7-425a-bca6-73cacde562a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:41.391199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 860/1725] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:41.392895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fd764-9601-49f1-99b5-bdbb55ed6b1d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:41.406494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124fd764-9601-49f1-99b5-bdbb55ed6b1d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:41.422990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb45e13-ff4c-4378-887f-aa3093f6f96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:41.425328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb45e13-ff4c-4378-887f-aa3093f6f96d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:41.425695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb45e13-ff4c-4378-887f-aa3093f6f96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:41.426229 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb45e13-ff4c-4378-887f-aa3093f6f96d admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:41.427858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb45e13-ff4c-4378-887f-aa3093f6f96d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:41.428914 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb45e13-ff4c-4378-887f-aa3093f6f96d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:41.437098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb45e13-ff4c-4378-887f-aa3093f6f96d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:41.500794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 866/1726] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:41.502057 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ea78f-d4a7-425a-bca6-73cacde562a5 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:41.522751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ea78f-d4a7-425a-bca6-73cacde562a5 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:41.566755 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0ab46-07f6-4d27-b65c-a5c3ca07d17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:41.568208 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0ab46-07f6-4d27-b65c-a5c3ca07d17e admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:41.568208 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0ab46-07f6-4d27-b65c-a5c3ca07d17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:41.568660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0ab46-07f6-4d27-b65c-a5c3ca07d17e admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:41.569087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0ab46-07f6-4d27-b65c-a5c3ca07d17e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:41.571118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0ab46-07f6-4d27-b65c-a5c3ca07d17e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:41.633390 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 861/1727] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:41 2022] POST /identity/v3/projects => generated 372 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:41.637576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb45e13-ff4c-4378-887f-aa3093f6f96d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:41.662529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb45e13-ff4c-4378-887f-aa3093f6f96d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:41.800901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a6750-7679-4f1c-83e8-e94a82b9da41 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:41.801285 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a6750-7679-4f1c-83e8-e94a82b9da41 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:41.801462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a6750-7679-4f1c-83e8-e94a82b9da41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:41.805576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d3d7e-269f-497b-a56e-97ac1c4966b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:41.809837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d3d7e-269f-497b-a56e-97ac1c4966b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:41.810619 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d3d7e-269f-497b-a56e-97ac1c4966b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:41.811254 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d3d7e-269f-497b-a56e-97ac1c4966b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:41.812163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d3d7e-269f-497b-a56e-97ac1c4966b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:41.815293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d3d7e-269f-497b-a56e-97ac1c4966b0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:41.832221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-424a6750-7679-4f1c-83e8-e94a82b9da41 None None] MFA Rules not processed for user `e8fc60410fab43a39bdb80478348ddae`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:41.833791 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-424a6750-7679-4f1c-83e8-e94a82b9da41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:41.845973 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:41.865207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 867/1728] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:41 2022] POST /identity/v3/users => generated 287 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:41.867105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d3d7e-269f-497b-a56e-97ac1c4966b0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:41.891333 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d3d7e-269f-497b-a56e-97ac1c4966b0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:41.903053 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fb4fa-d81c-42cb-9cfe-8a1985e29730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:41.903698 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb4fa-d81c-42cb-9cfe-8a1985e29730 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:41.903980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb4fa-d81c-42cb-9cfe-8a1985e29730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:41.904256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278fb4fa-d81c-42cb-9cfe-8a1985e29730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:41.904963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fb4fa-d81c-42cb-9cfe-8a1985e29730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:41.906574 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278fb4fa-d81c-42cb-9cfe-8a1985e29730 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:41.936853 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 868/1729] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:41 2022] GET /identity/v3/roles => generated 1427 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:41.940545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fb4fa-d81c-42cb-9cfe-8a1985e29730 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:41.962723 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278fb4fa-d81c-42cb-9cfe-8a1985e29730 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:41.964582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 862/1730] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:41.966365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29fe5f-8277-49d7-a5a9-a338d2285a4c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:41.966700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29fe5f-8277-49d7-a5a9-a338d2285a4c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:41.966956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29fe5f-8277-49d7-a5a9-a338d2285a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:41.973983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f0328-0473-4df2-bfc7-e9d013bf9844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:41.974721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f0328-0473-4df2-bfc7-e9d013bf9844 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:41.975065 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f0328-0473-4df2-bfc7-e9d013bf9844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:41.975340 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f0328-0473-4df2-bfc7-e9d013bf9844 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:41.976055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f0328-0473-4df2-bfc7-e9d013bf9844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:41.977671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f0328-0473-4df2-bfc7-e9d013bf9844 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:42.000071 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 869/1731] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:41 2022] GET /identity/v3/roles => generated 1427 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:42.004581 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f0328-0473-4df2-bfc7-e9d013bf9844 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:42.024353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-6c29fe5f-8277-49d7-a5a9-a338d2285a4c None None] MFA Rules not processed for user `e8fc60410fab43a39bdb80478348ddae`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:42.027700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6c29fe5f-8277-49d7-a5a9-a338d2285a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:42.032180 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f0328-0473-4df2-bfc7-e9d013bf9844 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:42.042514 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 863/1732] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:42.048633 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318728e-26d0-48d6-b05d-9d51952adf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:42.051793 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29fe5f-8277-49d7-a5a9-a338d2285a4c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:42.053732 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318728e-26d0-48d6-b05d-9d51952adf9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:42.054029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318728e-26d0-48d6-b05d-9d51952adf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:42.054311 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318728e-26d0-48d6-b05d-9d51952adf9b admin admin] PATH_INFO: `/v3/projects/73844e0c31cd447088deff1d9577555a/users/9365096e48c147348116efdd1521d1e8/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:42.065049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6c29fe5f-8277-49d7-a5a9-a338d2285a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:42.078831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318728e-26d0-48d6-b05d-9d51952adf9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=73844e0c31cd447088deff1d9577555a, user_id=9365096e48c147348116efdd1521d1e8, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:42.083254 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318728e-26d0-48d6-b05d-9d51952adf9b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:42.116511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 870/1733] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:42 2022] PUT /identity/v3/projects/73844e0c31cd447088deff1d9577555a/users/9365096e48c147348116efdd1521d1e8/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:42.128650 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cde6f-37c0-4288-91f5-e918e8cba710 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:42.128998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cde6f-37c0-4288-91f5-e918e8cba710 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:42.129282 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166cde6f-37c0-4288-91f5-e918e8cba710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:42.184089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-166cde6f-37c0-4288-91f5-e918e8cba710 None None] MFA Rules not processed for user `9365096e48c147348116efdd1521d1e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:42.185351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-166cde6f-37c0-4288-91f5-e918e8cba710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:42.224179 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29fe5f-8277-49d7-a5a9-a338d2285a4c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:42.367617 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:42.384109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 871/1734] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:42.395673 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166cde6f-37c0-4288-91f5-e918e8cba710 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:42.403244 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e796e3-32c8-4751-aa65-ccf32da8f1d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:42.403875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e796e3-32c8-4751-aa65-ccf32da8f1d1 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:42.405273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e796e3-32c8-4751-aa65-ccf32da8f1d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:42.405582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e796e3-32c8-4751-aa65-ccf32da8f1d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:42.502927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:42.520810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166cde6f-37c0-4288-91f5-e918e8cba710 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:42.536849 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3229ac-92d8-4af5-8d80-dad60cbf5e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:42.537765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3229ac-92d8-4af5-8d80-dad60cbf5e4f admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:42.543042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3229ac-92d8-4af5-8d80-dad60cbf5e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:42.543756 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3229ac-92d8-4af5-8d80-dad60cbf5e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:42.544707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3229ac-92d8-4af5-8d80-dad60cbf5e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:42.558883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e796e3-32c8-4751-aa65-ccf32da8f1d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:42.559138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3229ac-92d8-4af5-8d80-dad60cbf5e4f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:42.560593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e796e3-32c8-4751-aa65-ccf32da8f1d1 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:42.596125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 864/1735] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:42.597821 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786a838-7839-4f8a-826b-7b0ad134bb5c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:42.598028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786a838-7839-4f8a-826b-7b0ad134bb5c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:42.598394 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786a838-7839-4f8a-826b-7b0ad134bb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:42.631195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 872/1736] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:42 2022] POST /identity/v3/projects => generated 374 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:42.633833 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e65e3-bac2-4d2b-beb0-d60a8b6ad34e None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:42.634192 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e65e3-bac2-4d2b-beb0-d60a8b6ad34e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:42.634722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e65e3-bac2-4d2b-beb0-d60a8b6ad34e None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:42.640639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 873/1737] 10.222.0.100 () {54 vars in 926 bytes} [Mon Aug 1 15:14:42 2022] GET /identity => generated 271 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:14:42.641658 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e65e3-bac2-4d2b-beb0-d60a8b6ad34e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:42.655794 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0786a838-7839-4f8a-826b-7b0ad134bb5c None None] MFA Rules not processed for user `ca1d32ee23bc42b38ea9a043f7a850f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:42.659246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0786a838-7839-4f8a-826b-7b0ad134bb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:42.823191 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e65e3-bac2-4d2b-beb0-d60a8b6ad34e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:42.847481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b7466-22c2-4627-b74d-86162f14478b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:42.851764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b7466-22c2-4627-b74d-86162f14478b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:42.853435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b7466-22c2-4627-b74d-86162f14478b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:42.855008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b7466-22c2-4627-b74d-86162f14478b admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:42.856165 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b7466-22c2-4627-b74d-86162f14478b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:42.859857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b7466-22c2-4627-b74d-86162f14478b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:42.868706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 865/1738] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:42.872025 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a2cb4-5090-4b5a-9866-e88b8c8751a4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:42.872505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a2cb4-5090-4b5a-9866-e88b8c8751a4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:42.873337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a2cb4-5090-4b5a-9866-e88b8c8751a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:42.922489 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 874/1739] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:42 2022] POST /identity/v3/users => generated 294 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:42.935087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b22a2cb4-5090-4b5a-9866-e88b8c8751a4 None None] MFA Rules not processed for user `3bac5863e7bc4c7e854bd60912adb561`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:42.936389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b22a2cb4-5090-4b5a-9866-e88b8c8751a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:42.940344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddd4e5-3c3d-459c-b731-fef7b5ce81b7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:42.940896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddd4e5-3c3d-459c-b731-fef7b5ce81b7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:42.941380 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddd4e5-3c3d-459c-b731-fef7b5ce81b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:42.998459 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fbddd4e5-3c3d-459c-b731-fef7b5ce81b7 None None] MFA Rules not processed for user `ca1d32ee23bc42b38ea9a043f7a850f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:43.000528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fbddd4e5-3c3d-459c-b731-fef7b5ce81b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:43.018553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 875/1740] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:43.019480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddd4e5-3c3d-459c-b731-fef7b5ce81b7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:43.051189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddd4e5-3c3d-459c-b731-fef7b5ce81b7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:43.056755 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415dd961-d67a-4913-9ccd-23c7e86e82f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:43.057384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dd961-d67a-4913-9ccd-23c7e86e82f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:43.057663 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dd961-d67a-4913-9ccd-23c7e86e82f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:43.058458 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415dd961-d67a-4913-9ccd-23c7e86e82f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:43.059215 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415dd961-d67a-4913-9ccd-23c7e86e82f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:43.060859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415dd961-d67a-4913-9ccd-23c7e86e82f9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:43.095337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 876/1741] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:43 2022] GET /identity/v3/roles => generated 1427 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:43.099379 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415dd961-d67a-4913-9ccd-23c7e86e82f9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:43.103897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-415dd961-d67a-4913-9ccd-23c7e86e82f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:43.119515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 866/1742] 10.222.0.100 () {58 vars in 1023 bytes} [Mon Aug 1 15:14:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:43.120700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a2cb4-5090-4b5a-9866-e88b8c8751a4 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:43.122123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b22a2cb4-5090-4b5a-9866-e88b8c8751a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:43.132586 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415dd961-d67a-4913-9ccd-23c7e86e82f9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:43.143806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e34ea-24fd-4a0f-ae43-15b6a1e1b604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:43.144578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e34ea-24fd-4a0f-ae43-15b6a1e1b604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:43.144866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e34ea-24fd-4a0f-ae43-15b6a1e1b604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:43.145145 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e34ea-24fd-4a0f-ae43-15b6a1e1b604 admin admin] PATH_INFO: `/v3/users/ea431174273e48e5bdd7e2ab0a03f14f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:43.146469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e34ea-24fd-4a0f-ae43-15b6a1e1b604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea431174273e48e5bdd7e2ab0a03f14f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:43.148422 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e34ea-24fd-4a0f-ae43-15b6a1e1b604 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:43.248913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:43.250464 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-527e34ea-24fd-4a0f-ae43-15b6a1e1b604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea431174273e48e5bdd7e2ab0a03f14f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:43.251227 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-527e34ea-24fd-4a0f-ae43-15b6a1e1b604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea431174273e48e5bdd7e2ab0a03f14f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:43.263358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-527e34ea-24fd-4a0f-ae43-15b6a1e1b604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea431174273e48e5bdd7e2ab0a03f14f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:43.263863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-527e34ea-24fd-4a0f-ae43-15b6a1e1b604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea431174273e48e5bdd7e2ab0a03f14f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:43.287321 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-527e34ea-24fd-4a0f-ae43-15b6a1e1b604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea431174273e48e5bdd7e2ab0a03f14f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:43.288658 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:43.301337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a2cb4-5090-4b5a-9866-e88b8c8751a4 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:43.332794 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5e24a-ac69-4ed7-96d7-d19f045b33b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:43.333652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5e24a-ac69-4ed7-96d7-d19f045b33b9 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:43.334288 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5e24a-ac69-4ed7-96d7-d19f045b33b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:43.334727 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5e24a-ac69-4ed7-96d7-d19f045b33b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:43.341709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 877/1743] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:14:43 2022] DELETE /identity/v3/users/ea431174273e48e5bdd7e2ab0a03f14f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:43.341871 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-adb5e24a-ac69-4ed7-96d7-d19f045b33b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:43.346651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e34ea-24fd-4a0f-ae43-15b6a1e1b604 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:43.349357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-527e34ea-24fd-4a0f-ae43-15b6a1e1b604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:43.364566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5e24a-ac69-4ed7-96d7-d19f045b33b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:43.366943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5e24a-ac69-4ed7-96d7-d19f045b33b9 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:43.388604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 867/1744] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:43.389537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5e24a-ac69-4ed7-96d7-d19f045b33b9 service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:43.391001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-adb5e24a-ac69-4ed7-96d7-d19f045b33b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:43.428389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5e24a-ac69-4ed7-96d7-d19f045b33b9 service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:43.486180 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e34ea-24fd-4a0f-ae43-15b6a1e1b604 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:43.502846 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2b17e-f9df-49d8-b3c1-6fc9e8798ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:43.503816 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2b17e-f9df-49d8-b3c1-6fc9e8798ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:43.504186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2b17e-f9df-49d8-b3c1-6fc9e8798ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:43.504568 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2b17e-f9df-49d8-b3c1-6fc9e8798ed0 admin admin] PATH_INFO: `/v3/projects/013e45d9d09848048e6fcf02bb12c868/users/c83e1c7d5c52488cb85b14a64876ae45/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:43.585356 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2b17e-f9df-49d8-b3c1-6fc9e8798ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=013e45d9d09848048e6fcf02bb12c868, user_id=c83e1c7d5c52488cb85b14a64876ae45, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:43.588141 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2b17e-f9df-49d8-b3c1-6fc9e8798ed0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:43.640521 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 878/1745] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:43 2022] PUT /identity/v3/projects/013e45d9d09848048e6fcf02bb12c868/users/c83e1c7d5c52488cb85b14a64876ae45/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:43.644641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2b17e-f9df-49d8-b3c1-6fc9e8798ed0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:43.715591 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a941a7-129e-4e74-b175-0c1627860c96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:43.716253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a941a7-129e-4e74-b175-0c1627860c96 service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:43.716546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a941a7-129e-4e74-b175-0c1627860c96 service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:43.716835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a941a7-129e-4e74-b175-0c1627860c96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:43.734740 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-46a941a7-129e-4e74-b175-0c1627860c96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:43.770763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a941a7-129e-4e74-b175-0c1627860c96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:43.772633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a941a7-129e-4e74-b175-0c1627860c96 service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:43.803927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 868/1746] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:43.808072 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c9e0d-7212-48d1-806e-cb20d5227ad1 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:43.809065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c9e0d-7212-48d1-806e-cb20d5227ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:43.810215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c9e0d-7212-48d1-806e-cb20d5227ad1 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:43.814581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 869/1747] 10.222.0.100 () {54 vars in 917 bytes} [Mon Aug 1 15:14:43 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:14:43.819257 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb68631-6e01-444e-8b9f-bfaf23587045 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:43.819712 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb68631-6e01-444e-8b9f-bfaf23587045 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:43.820138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb68631-6e01-444e-8b9f-bfaf23587045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:43.879276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2b17e-f9df-49d8-b3c1-6fc9e8798ed0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:43.898747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3615b32f-13f3-4219-b33c-f97acf77c081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:43.900983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615b32f-13f3-4219-b33c-f97acf77c081 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:43.904150 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615b32f-13f3-4219-b33c-f97acf77c081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:43.905057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3615b32f-13f3-4219-b33c-f97acf77c081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:43.908385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615b32f-13f3-4219-b33c-f97acf77c081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:43.910882 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3615b32f-13f3-4219-b33c-f97acf77c081 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:43.970869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3fb68631-6e01-444e-8b9f-bfaf23587045 None None] MFA Rules not processed for user `e7f757eabcb84de6ab042e0ac5a8158c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:43.972300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3fb68631-6e01-444e-8b9f-bfaf23587045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:43.981237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 879/1748] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:43 2022] GET /identity/v3/roles => generated 1427 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:43.982959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104eaf0c-d5a0-4f56-8de5-77c875c8e682 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:43.989699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104eaf0c-d5a0-4f56-8de5-77c875c8e682 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:43.990688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104eaf0c-d5a0-4f56-8de5-77c875c8e682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:44.092461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-104eaf0c-d5a0-4f56-8de5-77c875c8e682 None None] MFA Rules not processed for user `f33ea26c1f72445bb4cb50ea503600dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:44.104668 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-104eaf0c-d5a0-4f56-8de5-77c875c8e682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:44.113008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 870/1749] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:14:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:44.115133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb68631-6e01-444e-8b9f-bfaf23587045 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:44.152814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb68631-6e01-444e-8b9f-bfaf23587045 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:44.164384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8cc39-debd-4984-a6c2-79f37a352644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:44.165577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8cc39-debd-4984-a6c2-79f37a352644 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:44.166089 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8cc39-debd-4984-a6c2-79f37a352644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:44.166589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8cc39-debd-4984-a6c2-79f37a352644 admin admin] PATH_INFO: `/v3/projects/013e45d9d09848048e6fcf02bb12c868/users/c83e1c7d5c52488cb85b14a64876ae45/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:44.173144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8cc39-debd-4984-a6c2-79f37a352644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=013e45d9d09848048e6fcf02bb12c868, user_id=c83e1c7d5c52488cb85b14a64876ae45, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:44.176903 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8cc39-debd-4984-a6c2-79f37a352644 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:44.218972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:44.221520 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 871/1750] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:44 2022] PUT /identity/v3/projects/013e45d9d09848048e6fcf02bb12c868/users/c83e1c7d5c52488cb85b14a64876ae45/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:44.225301 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8cc39-debd-4984-a6c2-79f37a352644 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:44.232524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4dc8cc39-debd-4984-a6c2-79f37a352644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:44.368596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 880/1751] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:44.370085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944a83d-9068-4183-8386-57d6f93cecfc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:44.370438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944a83d-9068-4183-8386-57d6f93cecfc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:44.371168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e944a83d-9068-4183-8386-57d6f93cecfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:44.399451 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8cc39-debd-4984-a6c2-79f37a352644 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:44.413119 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451810c-4c91-4dfe-8b74-de465016a0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:44.413823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451810c-4c91-4dfe-8b74-de465016a0d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:44.414083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451810c-4c91-4dfe-8b74-de465016a0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:44.415836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451810c-4c91-4dfe-8b74-de465016a0d4 admin admin] PATH_INFO: `/v3/projects/7517df1bfbef478a83f2a55e2a340799` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:44.418131 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451810c-4c91-4dfe-8b74-de465016a0d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7517df1bfbef478a83f2a55e2a340799)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:44.432375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451810c-4c91-4dfe-8b74-de465016a0d4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:44.436931 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:44.455218 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e944a83d-9068-4183-8386-57d6f93cecfc None None] MFA Rules not processed for user `c83e1c7d5c52488cb85b14a64876ae45`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:44.456494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e944a83d-9068-4183-8386-57d6f93cecfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:44.571097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f451810c-4c91-4dfe-8b74-de465016a0d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7517df1bfbef478a83f2a55e2a340799', 'request_id': 'req-f451810c-4c91-4dfe-8b74-de465016a0d4'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.583913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f451810c-4c91-4dfe-8b74-de465016a0d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7517df1bfbef478a83f2a55e2a340799', 'request_id': 'req-f451810c-4c91-4dfe-8b74-de465016a0d4'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.603991 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f451810c-4c91-4dfe-8b74-de465016a0d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7517df1bfbef478a83f2a55e2a340799', 'request_id': 'req-f451810c-4c91-4dfe-8b74-de465016a0d4'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.626170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 881/1752] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:44.627412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944a83d-9068-4183-8386-57d6f93cecfc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:44.627767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f451810c-4c91-4dfe-8b74-de465016a0d4 admin admin] The token cache is being invalidate because project 7517df1bfbef478a83f2a55e2a340799 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:44.628078 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f451810c-4c91-4dfe-8b74-de465016a0d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.630278 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f451810c-4c91-4dfe-8b74-de465016a0d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:44.633960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 872/1753] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:14:44 2022] DELETE /identity/v3/projects/7517df1bfbef478a83f2a55e2a340799 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:14:44.637420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa78923-302e-4fa8-acde-881b582f4c59 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:44.637970 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa78923-302e-4fa8-acde-881b582f4c59 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:44.638245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa78923-302e-4fa8-acde-881b582f4c59 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:44.640198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 873/1754] 10.222.0.100 () {54 vars in 917 bytes} [Mon Aug 1 15:14:44 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:14:44.645555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db144ac-53bb-44aa-8179-2e6fc99a781c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:44.645555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db144ac-53bb-44aa-8179-2e6fc99a781c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:44.645555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db144ac-53bb-44aa-8179-2e6fc99a781c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:44.655432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e944a83d-9068-4183-8386-57d6f93cecfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:44.681970 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-6db144ac-53bb-44aa-8179-2e6fc99a781c None None] MFA Rules not processed for user `424ccf230e16423a8dd32097d90c4995`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:44.684334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6db144ac-53bb-44aa-8179-2e6fc99a781c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:44.802286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 874/1755] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:44.804018 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db144ac-53bb-44aa-8179-2e6fc99a781c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:44.808380 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6db144ac-53bb-44aa-8179-2e6fc99a781c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:44.813746 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e944a83d-9068-4183-8386-57d6f93cecfc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:44.899985 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d441f-ab4b-4953-8399-2f13d50e811b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:44.900787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d441f-ab4b-4953-8399-2f13d50e811b service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:44.901108 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d441f-ab4b-4953-8399-2f13d50e811b service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:44.901384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d441f-ab4b-4953-8399-2f13d50e811b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:44.904205 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1a0d441f-ab4b-4953-8399-2f13d50e811b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:44.904715 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db144ac-53bb-44aa-8179-2e6fc99a781c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:44.910958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ae389-ea74-4cec-8a1b-feb1cdc413b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:44.912213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ae389-ea74-4cec-8a1b-feb1cdc413b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:44.913855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ae389-ea74-4cec-8a1b-feb1cdc413b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:44.914399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ae389-ea74-4cec-8a1b-feb1cdc413b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:44.916349 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d441f-ab4b-4953-8399-2f13d50e811b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:44.918115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d441f-ab4b-4953-8399-2f13d50e811b service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:44.921241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ef6ae389-ea74-4cec-8a1b-feb1cdc413b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:44.936531 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 882/1756] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:44.936860 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d441f-ab4b-4953-8399-2f13d50e811b service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:44.938112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1a0d441f-ab4b-4953-8399-2f13d50e811b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:44.963338 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:45.023170 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:45.039687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d441f-ab4b-4953-8399-2f13d50e811b service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:45.057222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0368d-9e6c-4a4b-983a-ad4b004a04d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:45.058804 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0368d-9e6c-4a4b-983a-ad4b004a04d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:45.059249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0368d-9e6c-4a4b-983a-ad4b004a04d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:45.059676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0368d-9e6c-4a4b-983a-ad4b004a04d5 admin admin] PATH_INFO: `/v3/users/8e0f46453e7948b180661239241d78f0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:45.067296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0368d-9e6c-4a4b-983a-ad4b004a04d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e0f46453e7948b180661239241d78f0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:45.069559 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0368d-9e6c-4a4b-983a-ad4b004a04d5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:45.072663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ae389-ea74-4cec-8a1b-feb1cdc413b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:45.074582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ae389-ea74-4cec-8a1b-feb1cdc413b7 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:45.100182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 875/1757] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:45.102513 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c2b8a-b6e9-4296-b673-4534f4ec787d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:45.106257 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c2b8a-b6e9-4296-b673-4534f4ec787d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:45.107072 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c2b8a-b6e9-4296-b673-4534f4ec787d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:45.165405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e5e0368d-9e6c-4a4b-983a-ad4b004a04d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e0f46453e7948b180661239241d78f0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:45.166198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e5e0368d-9e6c-4a4b-983a-ad4b004a04d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e0f46453e7948b180661239241d78f0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:45.168293 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7b9c2b8a-b6e9-4296-b673-4534f4ec787d None None] MFA Rules not processed for user `e7f757eabcb84de6ab042e0ac5a8158c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:45.169555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7b9c2b8a-b6e9-4296-b673-4534f4ec787d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:45.178791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e5e0368d-9e6c-4a4b-983a-ad4b004a04d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e0f46453e7948b180661239241d78f0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:45.179847 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e5e0368d-9e6c-4a4b-983a-ad4b004a04d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e0f46453e7948b180661239241d78f0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:45.181383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 876/1758] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:14:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:45.204947 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e5e0368d-9e6c-4a4b-983a-ad4b004a04d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e0f46453e7948b180661239241d78f0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:45.222245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c2b8a-b6e9-4296-b673-4534f4ec787d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:45.227490 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7b9c2b8a-b6e9-4296-b673-4534f4ec787d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:45.241788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c2b8a-b6e9-4296-b673-4534f4ec787d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:45.245608 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 883/1759] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:14:44 2022] DELETE /identity/v3/users/8e0f46453e7948b180661239241d78f0 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:45.414863 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c90a9a-ed28-4857-83be-1a91dee469c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:45.415466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c90a9a-ed28-4857-83be-1a91dee469c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:45.415753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c90a9a-ed28-4857-83be-1a91dee469c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:45.416195 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c90a9a-ed28-4857-83be-1a91dee469c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:45.418120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-87c90a9a-ed28-4857-83be-1a91dee469c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:45.442438 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c90a9a-ed28-4857-83be-1a91dee469c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:45.444025 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c90a9a-ed28-4857-83be-1a91dee469c6 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:45.466601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 877/1760] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:14:45.659320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0368d-9e6c-4a4b-983a-ad4b004a04d5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:45.661520 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e5e0368d-9e6c-4a4b-983a-ad4b004a04d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:45.797446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c90a9a-ed28-4857-83be-1a91dee469c6 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:45.799558 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-87c90a9a-ed28-4857-83be-1a91dee469c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:45.972046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0368d-9e6c-4a4b-983a-ad4b004a04d5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:45.987468 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c90a9a-ed28-4857-83be-1a91dee469c6 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:45.991479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5341eba-f872-47aa-a8c3-04256767ec3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:45.993850 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5341eba-f872-47aa-a8c3-04256767ec3e service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:45.994814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5341eba-f872-47aa-a8c3-04256767ec3e service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:45.995747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5341eba-f872-47aa-a8c3-04256767ec3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:45.999792 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b5341eba-f872-47aa-a8c3-04256767ec3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:46.000321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc285765-c362-4694-a4d6-ff3ea5a1e399 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:46.001248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc285765-c362-4694-a4d6-ff3ea5a1e399 service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:46.002358 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc285765-c362-4694-a4d6-ff3ea5a1e399 service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:46.003524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc285765-c362-4694-a4d6-ff3ea5a1e399 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:46.008533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fc285765-c362-4694-a4d6-ff3ea5a1e399 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:46.074792 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc285765-c362-4694-a4d6-ff3ea5a1e399 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:46.076903 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc285765-c362-4694-a4d6-ff3ea5a1e399 service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:46.107452 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 878/1761] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:14:46.108014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc285765-c362-4694-a4d6-ff3ea5a1e399 service placement] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:46.115811 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fc285765-c362-4694-a4d6-ff3ea5a1e399 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:46.124521 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:46.241864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5341eba-f872-47aa-a8c3-04256767ec3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:46.246188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5341eba-f872-47aa-a8c3-04256767ec3e service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:46.288541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 884/1762] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:46.289627 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5341eba-f872-47aa-a8c3-04256767ec3e service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:46.293594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b5341eba-f872-47aa-a8c3-04256767ec3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:46.348054 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:46.363606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc285765-c362-4694-a4d6-ff3ea5a1e399 service placement] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:46.383420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe0420-1d4c-43aa-8bd0-32e1b0e67aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:46.387016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe0420-1d4c-43aa-8bd0-32e1b0e67aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:46.387316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe0420-1d4c-43aa-8bd0-32e1b0e67aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:46.387585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe0420-1d4c-43aa-8bd0-32e1b0e67aa5 admin admin] PATH_INFO: `/v3/users/9365096e48c147348116efdd1521d1e8` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:46.393648 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe0420-1d4c-43aa-8bd0-32e1b0e67aa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9365096e48c147348116efdd1521d1e8)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:46.400441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe0420-1d4c-43aa-8bd0-32e1b0e67aa5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:46.501574 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5341eba-f872-47aa-a8c3-04256767ec3e service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:46.509550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e462b-25b4-48d6-8bab-316092972b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:46.510591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e462b-25b4-48d6-8bab-316092972b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:46.511195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e462b-25b4-48d6-8bab-316092972b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:46.511647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e462b-25b4-48d6-8bab-316092972b1a admin admin] PATH_INFO: `/v3/projects/7415bde6e4724d6aa960dc5bc35d3a69` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:46.514780 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e462b-25b4-48d6-8bab-316092972b1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7415bde6e4724d6aa960dc5bc35d3a69)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:46.517334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e462b-25b4-48d6-8bab-316092972b1a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:46.533542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b1fe0420-1d4c-43aa-8bd0-32e1b0e67aa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9365096e48c147348116efdd1521d1e8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:46.569901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b1fe0420-1d4c-43aa-8bd0-32e1b0e67aa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9365096e48c147348116efdd1521d1e8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:46.570676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b1fe0420-1d4c-43aa-8bd0-32e1b0e67aa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9365096e48c147348116efdd1521d1e8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:46.571377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b1fe0420-1d4c-43aa-8bd0-32e1b0e67aa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9365096e48c147348116efdd1521d1e8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:46.590153 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b1fe0420-1d4c-43aa-8bd0-32e1b0e67aa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9365096e48c147348116efdd1521d1e8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:46.606947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 879/1763] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:14:46 2022] DELETE /identity/v3/users/9365096e48c147348116efdd1521d1e8 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:46.608751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe0420-1d4c-43aa-8bd0-32e1b0e67aa5 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:46.611081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b1fe0420-1d4c-43aa-8bd0-32e1b0e67aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:46.653498 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a25e462b-25b4-48d6-8bab-316092972b1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7415bde6e4724d6aa960dc5bc35d3a69', 'request_id': 'req-a25e462b-25b4-48d6-8bab-316092972b1a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:46.668616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a25e462b-25b4-48d6-8bab-316092972b1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7415bde6e4724d6aa960dc5bc35d3a69', 'request_id': 'req-a25e462b-25b4-48d6-8bab-316092972b1a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:46.697840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a25e462b-25b4-48d6-8bab-316092972b1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7415bde6e4724d6aa960dc5bc35d3a69', 'request_id': 'req-a25e462b-25b4-48d6-8bab-316092972b1a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:46.737192 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a25e462b-25b4-48d6-8bab-316092972b1a admin admin] The token cache is being invalidate because project 7415bde6e4724d6aa960dc5bc35d3a69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:46.738846 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a25e462b-25b4-48d6-8bab-316092972b1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:46.745486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a25e462b-25b4-48d6-8bab-316092972b1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:46.757115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 885/1764] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:14:46 2022] DELETE /identity/v3/projects/7415bde6e4724d6aa960dc5bc35d3a69 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:46.770420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebf293-f7d4-487c-b6e2-460515340e5c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:46.773616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebf293-f7d4-487c-b6e2-460515340e5c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:46.775952 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebf293-f7d4-487c-b6e2-460515340e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:46.825593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe0420-1d4c-43aa-8bd0-32e1b0e67aa5 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:46.842229 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d7ebf293-f7d4-487c-b6e2-460515340e5c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:46.850519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d7ebf293-f7d4-487c-b6e2-460515340e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:47.075772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63743c92-f6d9-4571-8606-e82d2855aed3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:47.076444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63743c92-f6d9-4571-8606-e82d2855aed3 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:47.076824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63743c92-f6d9-4571-8606-e82d2855aed3 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:47.077273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63743c92-f6d9-4571-8606-e82d2855aed3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:47.079899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-63743c92-f6d9-4571-8606-e82d2855aed3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:47.094044 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:47.105167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 886/1765] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:47.114447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef844f05-94a9-420e-9054-3ff004865c72 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:47.114715 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef844f05-94a9-420e-9054-3ff004865c72 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:47.117841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef844f05-94a9-420e-9054-3ff004865c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:47.145403 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:47.180214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ef844f05-94a9-420e-9054-3ff004865c72 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:47.183479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ef844f05-94a9-420e-9054-3ff004865c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:47.212619 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 887/1766] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:47.221126 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef844f05-94a9-420e-9054-3ff004865c72 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:47.221439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ef844f05-94a9-420e-9054-3ff004865c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:47.255318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63743c92-f6d9-4571-8606-e82d2855aed3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:47.258409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63743c92-f6d9-4571-8606-e82d2855aed3 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:47.295364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 880/1767] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:47.299873 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fac85-e652-4913-8618-9defe58d75a6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:47.299873 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fac85-e652-4913-8618-9defe58d75a6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:47.300187 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6fac85-e652-4913-8618-9defe58d75a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:47.366362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2c6fac85-e652-4913-8618-9defe58d75a6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:47.382507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2c6fac85-e652-4913-8618-9defe58d75a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:47.441344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef844f05-94a9-420e-9054-3ff004865c72 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:47.447297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 881/1768] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:47.458639 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6fac85-e652-4913-8618-9defe58d75a6 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:47.459448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481be266-16de-4adc-8c8b-f4216286a8a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:47.464261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481be266-16de-4adc-8c8b-f4216286a8a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:47.464560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481be266-16de-4adc-8c8b-f4216286a8a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:47.469103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481be266-16de-4adc-8c8b-f4216286a8a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:47.501577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6fac85-e652-4913-8618-9defe58d75a6 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:47.514292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481be266-16de-4adc-8c8b-f4216286a8a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:47.522697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481be266-16de-4adc-8c8b-f4216286a8a7 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:47.543253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e55d0-5e3b-4c17-896d-bd9775d36be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:47.549833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e55d0-5e3b-4c17-896d-bd9775d36be7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:47.550153 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e55d0-5e3b-4c17-896d-bd9775d36be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:47.550435 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8e55d0-5e3b-4c17-896d-bd9775d36be7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:47.553795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e55d0-5e3b-4c17-896d-bd9775d36be7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:47.559068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e55d0-5e3b-4c17-896d-bd9775d36be7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:47.565462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8e55d0-5e3b-4c17-896d-bd9775d36be7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:47.578010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 888/1769] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:47.617706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 882/1770] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:47.619007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481be266-16de-4adc-8c8b-f4216286a8a7 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:47.639233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481be266-16de-4adc-8c8b-f4216286a8a7 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:47.653996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36357fce-acef-4cc0-9d55-d048d0814d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:47.654810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36357fce-acef-4cc0-9d55-d048d0814d3c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:47.655102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36357fce-acef-4cc0-9d55-d048d0814d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:47.655387 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36357fce-acef-4cc0-9d55-d048d0814d3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:47.656078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36357fce-acef-4cc0-9d55-d048d0814d3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:47.658453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36357fce-acef-4cc0-9d55-d048d0814d3c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:47.783867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 889/1771] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:47 2022] POST /identity/v3/projects => generated 382 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:47.789189 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e55d0-5e3b-4c17-896d-bd9775d36be7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:47.802928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2326fa-3262-4684-a9c6-f95ce2c9fb32 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:47.803483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2326fa-3262-4684-a9c6-f95ce2c9fb32 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:47.803909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2326fa-3262-4684-a9c6-f95ce2c9fb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:47.807092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8e55d0-5e3b-4c17-896d-bd9775d36be7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:47.888584 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-cc2326fa-3262-4684-a9c6-f95ce2c9fb32 None None] MFA Rules not processed for user `ca1d32ee23bc42b38ea9a043f7a850f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:47.890525 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cc2326fa-3262-4684-a9c6-f95ce2c9fb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:48.076320 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c9942-4ddb-401c-827e-205e4d6bf8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:48.078198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c9942-4ddb-401c-827e-205e4d6bf8de admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:48.081246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c9942-4ddb-401c-827e-205e4d6bf8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:48.082093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c9942-4ddb-401c-827e-205e4d6bf8de admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:48.083499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c9942-4ddb-401c-827e-205e4d6bf8de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:48.093795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c9942-4ddb-401c-827e-205e4d6bf8de admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:48.149617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 883/1772] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:47 2022] POST /identity/v3/users => generated 292 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:48.153991 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c9942-4ddb-401c-827e-205e4d6bf8de admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:48.165522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 890/1773] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:48.218603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c9942-4ddb-401c-827e-205e4d6bf8de admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:48.251259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedf096-ec30-490a-9575-8e5d0c1b4075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:48.252120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedf096-ec30-490a-9575-8e5d0c1b4075 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:48.254031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedf096-ec30-490a-9575-8e5d0c1b4075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:48.254720 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedf096-ec30-490a-9575-8e5d0c1b4075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:48.259512 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdedf096-ec30-490a-9575-8e5d0c1b4075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:48.265609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdedf096-ec30-490a-9575-8e5d0c1b4075 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:48.343828 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 884/1774] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:48 2022] GET /identity/v3/roles => generated 1427 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:48.347562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2326fa-3262-4684-a9c6-f95ce2c9fb32 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:48.379266 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2326fa-3262-4684-a9c6-f95ce2c9fb32 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:48.394305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bd16f9-1f24-47df-add7-974f1c2c1a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:48.395416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd16f9-1f24-47df-add7-974f1c2c1a8a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:48.395907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd16f9-1f24-47df-add7-974f1c2c1a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:48.396612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd16f9-1f24-47df-add7-974f1c2c1a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:48.406644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd16f9-1f24-47df-add7-974f1c2c1a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:48.408361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bd16f9-1f24-47df-add7-974f1c2c1a8a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:48.444030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 891/1775] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:48 2022] GET /identity/v3/roles => generated 1427 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:48.448629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedf096-ec30-490a-9575-8e5d0c1b4075 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:48.463508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedf096-ec30-490a-9575-8e5d0c1b4075 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:48.469870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f085f9e-95c5-407a-a297-5847f630f0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:48.471607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f085f9e-95c5-407a-a297-5847f630f0df admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:48.472005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f085f9e-95c5-407a-a297-5847f630f0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:48.472564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f085f9e-95c5-407a-a297-5847f630f0df admin admin] PATH_INFO: `/v3/projects/922ec89747ef43d0ab18b0e7ef7b5b39/users/7f02f57d4ea04ec4a92f76e35afe6592/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:48.512425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f085f9e-95c5-407a-a297-5847f630f0df admin admin] RBAC: Authorizing `identity:create_grant(project_id=922ec89747ef43d0ab18b0e7ef7b5b39, user_id=7f02f57d4ea04ec4a92f76e35afe6592, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:48.524591 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f085f9e-95c5-407a-a297-5847f630f0df admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:48.573834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 885/1776] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:48 2022] PUT /identity/v3/projects/922ec89747ef43d0ab18b0e7ef7b5b39/users/7f02f57d4ea04ec4a92f76e35afe6592/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:48.583944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876acd46-5119-4b24-bde2-d8f4eaf30696 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:48.584514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876acd46-5119-4b24-bde2-d8f4eaf30696 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:48.584994 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876acd46-5119-4b24-bde2-d8f4eaf30696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:48.656000 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-876acd46-5119-4b24-bde2-d8f4eaf30696 None None] MFA Rules not processed for user `7f02f57d4ea04ec4a92f76e35afe6592`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:48.664875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-876acd46-5119-4b24-bde2-d8f4eaf30696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:48.845893 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:48.858055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 892/1777] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:48.859415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f085f9e-95c5-407a-a297-5847f630f0df admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:48.934595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:48.977864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f085f9e-95c5-407a-a297-5847f630f0df admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:48.990741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bf10d-4db8-4271-972f-9e309bd0c1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:48.991311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bf10d-4db8-4271-972f-9e309bd0c1e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:48.991593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bf10d-4db8-4271-972f-9e309bd0c1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:48.991877 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bf10d-4db8-4271-972f-9e309bd0c1e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:48.992569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bf10d-4db8-4271-972f-9e309bd0c1e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:48.994652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bf10d-4db8-4271-972f-9e309bd0c1e7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:49.098538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 886/1778] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:48 2022] POST /identity/v3/projects => generated 378 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:49.104442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876acd46-5119-4b24-bde2-d8f4eaf30696 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:49.119512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876acd46-5119-4b24-bde2-d8f4eaf30696 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:49.191525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bf10d-4db8-4271-972f-9e309bd0c1e7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:49.194607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e79bf10d-4db8-4271-972f-9e309bd0c1e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:49.275499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6d157-21f6-475a-9b1d-ed50ef9ac132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:49.284365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6d157-21f6-475a-9b1d-ed50ef9ac132 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:49.284620 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6d157-21f6-475a-9b1d-ed50ef9ac132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:49.285554 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6d157-21f6-475a-9b1d-ed50ef9ac132 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:49.285679 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6d157-21f6-475a-9b1d-ed50ef9ac132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:49.290722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6d157-21f6-475a-9b1d-ed50ef9ac132 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:49.382712 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 893/1779] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:49 2022] POST /identity/v3/users => generated 296 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:49.390888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6d157-21f6-475a-9b1d-ed50ef9ac132 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:49.406408 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6d157-21f6-475a-9b1d-ed50ef9ac132 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:49.411891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e6f22-67b9-405b-9feb-2773e9aa1240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:49.412697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e6f22-67b9-405b-9feb-2773e9aa1240 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:49.413123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e6f22-67b9-405b-9feb-2773e9aa1240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:49.413541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e6f22-67b9-405b-9feb-2773e9aa1240 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:49.414786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e6f22-67b9-405b-9feb-2773e9aa1240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:49.416932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e6f22-67b9-405b-9feb-2773e9aa1240 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:49.419497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bf10d-4db8-4271-972f-9e309bd0c1e7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:49.429550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 894/1780] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:49 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:49.435257 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e6f22-67b9-405b-9feb-2773e9aa1240 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:49.440981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:49.443137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:49.443541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:49.443831 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] PATH_INFO: `/v3/projects/73844e0c31cd447088deff1d9577555a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:49.449376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73844e0c31cd447088deff1d9577555a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:49.456060 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:49.459329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e6f22-67b9-405b-9feb-2773e9aa1240 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:49.468416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc92bb-94ec-4a05-bccd-391144fbd980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:49.470390 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc92bb-94ec-4a05-bccd-391144fbd980 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:49.470704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc92bb-94ec-4a05-bccd-391144fbd980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:49.470995 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc92bb-94ec-4a05-bccd-391144fbd980 admin admin] PATH_INFO: `/v3/projects/bccaadc383f845bfa8247425217facfa/users/20df6d114ac24ea497df923011007f1b/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:49.499237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc92bb-94ec-4a05-bccd-391144fbd980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccaadc383f845bfa8247425217facfa, user_id=20df6d114ac24ea497df923011007f1b, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:49.502286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc92bb-94ec-4a05-bccd-391144fbd980 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:49.525258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 895/1781] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:49 2022] PUT /identity/v3/projects/bccaadc383f845bfa8247425217facfa/users/20df6d114ac24ea497df923011007f1b/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:49.530049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc92bb-94ec-4a05-bccd-391144fbd980 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:49.639597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73844e0c31cd447088deff1d9577555a', 'request_id': 'req-9af95981-3227-4929-9e56-8f710058c5b6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:49.670385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73844e0c31cd447088deff1d9577555a', 'request_id': 'req-9af95981-3227-4929-9e56-8f710058c5b6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:49.698202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73844e0c31cd447088deff1d9577555a', 'request_id': 'req-9af95981-3227-4929-9e56-8f710058c5b6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:49.730638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] The token cache is being invalidate because project 73844e0c31cd447088deff1d9577555a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:49.730960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:49.731603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:49.733630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 887/1782] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:14:49 2022] DELETE /identity/v3/projects/73844e0c31cd447088deff1d9577555a => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:49.737486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:49.739326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:49.767469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc92bb-94ec-4a05-bccd-391144fbd980 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:49.994346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af95981-3227-4929-9e56-8f710058c5b6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:50.009652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88faf805-40a8-4f22-bb85-1bb10ceafb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:50.013021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faf805-40a8-4f22-bb85-1bb10ceafb1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:50.013518 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faf805-40a8-4f22-bb85-1bb10ceafb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:50.014552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faf805-40a8-4f22-bb85-1bb10ceafb1d admin admin] PATH_INFO: `/v3/users/c83e1c7d5c52488cb85b14a64876ae45` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:50.022594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88faf805-40a8-4f22-bb85-1bb10ceafb1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c83e1c7d5c52488cb85b14a64876ae45)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:50.036041 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88faf805-40a8-4f22-bb85-1bb10ceafb1d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:50.046741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:50.064958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fb97c-c9ed-4e2d-89d0-becccdcb74c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:50.068932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fb97c-c9ed-4e2d-89d0-becccdcb74c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:50.069222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fb97c-c9ed-4e2d-89d0-becccdcb74c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:50.072581 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fb97c-c9ed-4e2d-89d0-becccdcb74c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:50.073471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fb97c-c9ed-4e2d-89d0-becccdcb74c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:50.077593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fb97c-c9ed-4e2d-89d0-becccdcb74c0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:50.087450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:50.120709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 896/1783] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:49 2022] GET /identity/v3/roles => generated 1427 bytes in 590 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:50.121771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fb97c-c9ed-4e2d-89d0-becccdcb74c0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:50.127147 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ec6fb97c-c9ed-4e2d-89d0-becccdcb74c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:50.164172 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-88faf805-40a8-4f22-bb85-1bb10ceafb1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c83e1c7d5c52488cb85b14a64876ae45'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:50.194139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fb97c-c9ed-4e2d-89d0-becccdcb74c0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:50.229085 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-88faf805-40a8-4f22-bb85-1bb10ceafb1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c83e1c7d5c52488cb85b14a64876ae45'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:50.232476 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-88faf805-40a8-4f22-bb85-1bb10ceafb1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c83e1c7d5c52488cb85b14a64876ae45'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:50.234479 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-88faf805-40a8-4f22-bb85-1bb10ceafb1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c83e1c7d5c52488cb85b14a64876ae45'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:50.237118 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648ec1b-ca22-4e70-867b-ad15eba1c090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:50.238103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648ec1b-ca22-4e70-867b-ad15eba1c090 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:50.238103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648ec1b-ca22-4e70-867b-ad15eba1c090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:50.238588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648ec1b-ca22-4e70-867b-ad15eba1c090 admin admin] PATH_INFO: `/v3/projects/bccaadc383f845bfa8247425217facfa/users/20df6d114ac24ea497df923011007f1b/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:50.263213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648ec1b-ca22-4e70-867b-ad15eba1c090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bccaadc383f845bfa8247425217facfa, user_id=20df6d114ac24ea497df923011007f1b, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:50.278075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648ec1b-ca22-4e70-867b-ad15eba1c090 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:50.318350 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-88faf805-40a8-4f22-bb85-1bb10ceafb1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c83e1c7d5c52488cb85b14a64876ae45'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:50.335653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 897/1784] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:50 2022] PUT /identity/v3/projects/bccaadc383f845bfa8247425217facfa/users/20df6d114ac24ea497df923011007f1b/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:50.347481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738b07d-b701-4d55-946d-b9e3f5e73a85 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:50.347785 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738b07d-b701-4d55-946d-b9e3f5e73a85 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:50.348048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1738b07d-b701-4d55-946d-b9e3f5e73a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:50.390120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 888/1785] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:14:49 2022] DELETE /identity/v3/users/c83e1c7d5c52488cb85b14a64876ae45 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:50.451286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1738b07d-b701-4d55-946d-b9e3f5e73a85 None None] MFA Rules not processed for user `20df6d114ac24ea497df923011007f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:50.454169 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1738b07d-b701-4d55-946d-b9e3f5e73a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:50.719244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 898/1786] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:14:50.731246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88faf805-40a8-4f22-bb85-1bb10ceafb1d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:50.733564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-88faf805-40a8-4f22-bb85-1bb10ceafb1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:50.949984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88faf805-40a8-4f22-bb85-1bb10ceafb1d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:50.960223 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611eaf70-4f7d-40d9-87ff-a5c178ad6fb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:50.961171 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eaf70-4f7d-40d9-87ff-a5c178ad6fb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:50.961465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eaf70-4f7d-40d9-87ff-a5c178ad6fb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:50.962209 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611eaf70-4f7d-40d9-87ff-a5c178ad6fb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:50.987592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611eaf70-4f7d-40d9-87ff-a5c178ad6fb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:50.989106 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611eaf70-4f7d-40d9-87ff-a5c178ad6fb9 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:51.028234 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 889/1787] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:51.143731 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738b07d-b701-4d55-946d-b9e3f5e73a85 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:51.145269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1738b07d-b701-4d55-946d-b9e3f5e73a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:51.256497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611eaf70-4f7d-40d9-87ff-a5c178ad6fb9 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:51.271480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611eaf70-4f7d-40d9-87ff-a5c178ad6fb9 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:51.286601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b195a87-d0ee-4bc6-bf5f-6b75b89175ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:51.287264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b195a87-d0ee-4bc6-bf5f-6b75b89175ab service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:51.294240 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b195a87-d0ee-4bc6-bf5f-6b75b89175ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:51.294705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b195a87-d0ee-4bc6-bf5f-6b75b89175ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:51.301919 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2b195a87-d0ee-4bc6-bf5f-6b75b89175ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:51.467925 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1738b07d-b701-4d55-946d-b9e3f5e73a85 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:51.483394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff40ce0-1d59-43ea-94b2-516484909c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:51.484068 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff40ce0-1d59-43ea-94b2-516484909c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:51.484360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff40ce0-1d59-43ea-94b2-516484909c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:51.484624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff40ce0-1d59-43ea-94b2-516484909c1c admin admin] PATH_INFO: `/v3/users/0944cc7d776f42fb9e11eea3cc193509` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:51.494299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff40ce0-1d59-43ea-94b2-516484909c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0944cc7d776f42fb9e11eea3cc193509)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:51.498504 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff40ce0-1d59-43ea-94b2-516484909c1c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:51.571534 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b195a87-d0ee-4bc6-bf5f-6b75b89175ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:51.583351 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b195a87-d0ee-4bc6-bf5f-6b75b89175ab service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:51.630368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 890/1788] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:51.646510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9ff40ce0-1d59-43ea-94b2-516484909c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0944cc7d776f42fb9e11eea3cc193509'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:51.660046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9ff40ce0-1d59-43ea-94b2-516484909c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0944cc7d776f42fb9e11eea3cc193509'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:51.681539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9ff40ce0-1d59-43ea-94b2-516484909c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0944cc7d776f42fb9e11eea3cc193509'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:51.688217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9ff40ce0-1d59-43ea-94b2-516484909c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0944cc7d776f42fb9e11eea3cc193509'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:51.749793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:51.754589 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9ff40ce0-1d59-43ea-94b2-516484909c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0944cc7d776f42fb9e11eea3cc193509'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:51.809744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 899/1789] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:14:51 2022] DELETE /identity/v3/users/0944cc7d776f42fb9e11eea3cc193509 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:51.818638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125fec0-339e-4df4-9997-b57601b59fc3 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:51.819215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125fec0-339e-4df4-9997-b57601b59fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:51.820120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125fec0-339e-4df4-9997-b57601b59fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:51.856792 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10845a0a-6b7d-4af9-9769-c99b2a6dcfd0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:51.857088 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10845a0a-6b7d-4af9-9769-c99b2a6dcfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:51.857371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10845a0a-6b7d-4af9-9769-c99b2a6dcfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:51.893091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-8125fec0-339e-4df4-9997-b57601b59fc3 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:51.896359 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8125fec0-339e-4df4-9997-b57601b59fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:51.952285 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-10845a0a-6b7d-4af9-9769-c99b2a6dcfd0 None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:51.953820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-10845a0a-6b7d-4af9-9769-c99b2a6dcfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:51.955872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:52.123636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 891/1790] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:52.164195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 900/1791] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:14:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 01 15:14:52.265472 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125fec0-339e-4df4-9997-b57601b59fc3 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:52.270317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8125fec0-339e-4df4-9997-b57601b59fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:52.430549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125fec0-339e-4df4-9997-b57601b59fc3 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:52.438055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1859f6d9-d265-4b46-9e3f-47698d1a6ada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:52.438848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859f6d9-d265-4b46-9e3f-47698d1a6ada service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:52.439210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859f6d9-d265-4b46-9e3f-47698d1a6ada service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:52.439629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1859f6d9-d265-4b46-9e3f-47698d1a6ada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:52.444283 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10845a0a-6b7d-4af9-9769-c99b2a6dcfd0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:52.450308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-10845a0a-6b7d-4af9-9769-c99b2a6dcfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:52.490725 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859f6d9-d265-4b46-9e3f-47698d1a6ada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:52.496698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1859f6d9-d265-4b46-9e3f-47698d1a6ada service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:52.536557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 892/1792] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:52.652821 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10845a0a-6b7d-4af9-9769-c99b2a6dcfd0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:52.675064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4313d40-adc6-4cb8-9d1a-8482e749cfd8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:52.676180 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4313d40-adc6-4cb8-9d1a-8482e749cfd8 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:52.676599 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4313d40-adc6-4cb8-9d1a-8482e749cfd8 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:52.677603 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4313d40-adc6-4cb8-9d1a-8482e749cfd8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:52.692814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4313d40-adc6-4cb8-9d1a-8482e749cfd8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:52.695594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4313d40-adc6-4cb8-9d1a-8482e749cfd8 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:52.705463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 901/1793] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:14:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:14:52.733614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf68cf-294f-4fae-b121-f514b72e1e8d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:52.734122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf68cf-294f-4fae-b121-f514b72e1e8d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:52.734504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf68cf-294f-4fae-b121-f514b72e1e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:52.774942 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f2bf68cf-294f-4fae-b121-f514b72e1e8d None None] MFA Rules not processed for user `424ccf230e16423a8dd32097d90c4995`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:52.778862 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f2bf68cf-294f-4fae-b121-f514b72e1e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:53.081794 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 893/1794] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:54.534348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4313d40-adc6-4cb8-9d1a-8482e749cfd8 service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:54.536260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e4313d40-adc6-4cb8-9d1a-8482e749cfd8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:54.577317 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4313d40-adc6-4cb8-9d1a-8482e749cfd8 service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:54.599273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81493ad-6a12-412b-bf43-e3d4faee362b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:54.600188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81493ad-6a12-412b-bf43-e3d4faee362b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:54.600636 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81493ad-6a12-412b-bf43-e3d4faee362b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:54.601085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81493ad-6a12-412b-bf43-e3d4faee362b admin admin] PATH_INFO: `/v3/projects/013e45d9d09848048e6fcf02bb12c868` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:54.607649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81493ad-6a12-412b-bf43-e3d4faee362b admin admin] RBAC: Authorizing `identity:delete_project(project_id=013e45d9d09848048e6fcf02bb12c868)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:54.610205 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81493ad-6a12-412b-bf43-e3d4faee362b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:54.728782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:54.789598 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b81493ad-6a12-412b-bf43-e3d4faee362b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '013e45d9d09848048e6fcf02bb12c868', 'request_id': 'req-b81493ad-6a12-412b-bf43-e3d4faee362b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:54.810359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b81493ad-6a12-412b-bf43-e3d4faee362b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '013e45d9d09848048e6fcf02bb12c868', 'request_id': 'req-b81493ad-6a12-412b-bf43-e3d4faee362b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:54.857864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b81493ad-6a12-412b-bf43-e3d4faee362b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '013e45d9d09848048e6fcf02bb12c868', 'request_id': 'req-b81493ad-6a12-412b-bf43-e3d4faee362b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:54.910964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b81493ad-6a12-412b-bf43-e3d4faee362b admin admin] The token cache is being invalidate because project 013e45d9d09848048e6fcf02bb12c868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:54.912396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b81493ad-6a12-412b-bf43-e3d4faee362b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:54.918029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b81493ad-6a12-412b-bf43-e3d4faee362b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:54.928283 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 902/1795] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:14:54 2022] DELETE /identity/v3/projects/013e45d9d09848048e6fcf02bb12c868 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:54.933504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcb1fa-6159-4993-9e3e-ceb8104f1dc6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:54.934328 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcb1fa-6159-4993-9e3e-ceb8104f1dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:54.934328 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcb1fa-6159-4993-9e3e-ceb8104f1dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:54.983077 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3cdcb1fa-6159-4993-9e3e-ceb8104f1dc6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:54.986805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3cdcb1fa-6159-4993-9e3e-ceb8104f1dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:55.170813 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 894/1796] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:55.184907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d704d-9a5e-46a9-97ff-78a502e4c989 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:55.185710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d704d-9a5e-46a9-97ff-78a502e4c989 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:55.186341 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d704d-9a5e-46a9-97ff-78a502e4c989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:55.255729 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6d5d704d-9a5e-46a9-97ff-78a502e4c989 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:55.261693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6d5d704d-9a5e-46a9-97ff-78a502e4c989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:55.317720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 903/1797] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:55.318935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcb1fa-6159-4993-9e3e-ceb8104f1dc6 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:55.341874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcb1fa-6159-4993-9e3e-ceb8104f1dc6 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:55.352563 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fe99b-082f-491f-9b59-5c79dfd1ce44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:55.353297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fe99b-082f-491f-9b59-5c79dfd1ce44 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:55.353578 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fe99b-082f-491f-9b59-5c79dfd1ce44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:55.353897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fe99b-082f-491f-9b59-5c79dfd1ce44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:55.354628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fe99b-082f-491f-9b59-5c79dfd1ce44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:55.359905 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fe99b-082f-491f-9b59-5c79dfd1ce44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:55.361857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fe99b-082f-491f-9b59-5c79dfd1ce44 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:55.372624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:55.383580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d704d-9a5e-46a9-97ff-78a502e4c989 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:55.391004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 895/1798] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:14:55.395871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d704d-9a5e-46a9-97ff-78a502e4c989 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:55.402820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754439b8-0611-49a1-83e0-e2be3a60a7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:55.403647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754439b8-0611-49a1-83e0-e2be3a60a7ef admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:55.404153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754439b8-0611-49a1-83e0-e2be3a60a7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:55.404549 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754439b8-0611-49a1-83e0-e2be3a60a7ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:55.405575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754439b8-0611-49a1-83e0-e2be3a60a7ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:55.407897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754439b8-0611-49a1-83e0-e2be3a60a7ef admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:55.480339 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 904/1799] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:55 2022] POST /identity/v3/projects => generated 368 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:14:55.486086 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fe99b-082f-491f-9b59-5c79dfd1ce44 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:55.495263 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754439b8-0611-49a1-83e0-e2be3a60a7ef admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:55.502257 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-754439b8-0611-49a1-83e0-e2be3a60a7ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:55.537334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fe99b-082f-491f-9b59-5c79dfd1ce44 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:55.813369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e50f0-d648-47bd-9a15-b7bf65a2c3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:55.815930 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e50f0-d648-47bd-9a15-b7bf65a2c3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:55.820499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e50f0-d648-47bd-9a15-b7bf65a2c3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:55.820963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e50f0-d648-47bd-9a15-b7bf65a2c3b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:55.825290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e50f0-d648-47bd-9a15-b7bf65a2c3b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:55.834577 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754439b8-0611-49a1-83e0-e2be3a60a7ef admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:55.835904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e50f0-d648-47bd-9a15-b7bf65a2c3b9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:55.862564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:55.863451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:55.863981 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:55.864839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] PATH_INFO: `/v3/projects/075ab2f308204d83b18d745778bbdef7` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:55.866861 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=075ab2f308204d83b18d745778bbdef7)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:55.869048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:55.899370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 896/1800] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:55 2022] POST /identity/v3/users => generated 285 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:55.904584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e50f0-d648-47bd-9a15-b7bf65a2c3b9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:55.930229 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e50f0-d648-47bd-9a15-b7bf65a2c3b9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:55.951261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1dc589-8d12-4ef1-836f-25ce73741698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:55.952133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1dc589-8d12-4ef1-836f-25ce73741698 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:55.953003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1dc589-8d12-4ef1-836f-25ce73741698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:55.953513 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1dc589-8d12-4ef1-836f-25ce73741698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:55.954461 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1dc589-8d12-4ef1-836f-25ce73741698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:55.961718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1dc589-8d12-4ef1-836f-25ce73741698 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:55.997605 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '075ab2f308204d83b18d745778bbdef7', 'request_id': 'req-2c0b2ae8-7246-4636-8ab6-ad94476233e8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:56.008405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '075ab2f308204d83b18d745778bbdef7', 'request_id': 'req-2c0b2ae8-7246-4636-8ab6-ad94476233e8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:56.017886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '075ab2f308204d83b18d745778bbdef7', 'request_id': 'req-2c0b2ae8-7246-4636-8ab6-ad94476233e8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:56.028760 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 897/1801] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:55 2022] GET /identity/v3/roles => generated 1427 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:56.032154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1dc589-8d12-4ef1-836f-25ce73741698 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:56.058559 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] The token cache is being invalidate because project 075ab2f308204d83b18d745778bbdef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:56.059028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:56.059963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:56.065431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 905/1802] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:14:55 2022] DELETE /identity/v3/projects/075ab2f308204d83b18d745778bbdef7 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:56.067078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:56.069874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:56.099080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1dc589-8d12-4ef1-836f-25ce73741698 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:56.179904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:56.259009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2c0151-174c-4cfc-a1b2-adaedba99edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:56.260543 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c0151-174c-4cfc-a1b2-adaedba99edf admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:56.260973 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c0151-174c-4cfc-a1b2-adaedba99edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:56.262416 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c0151-174c-4cfc-a1b2-adaedba99edf admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:56.264387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c0151-174c-4cfc-a1b2-adaedba99edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:56.267732 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2c0151-174c-4cfc-a1b2-adaedba99edf admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:56.287100 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 898/1803] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:56 2022] GET /identity/v3/roles => generated 1427 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:56.288814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a492d-6140-4744-8147-933ea3be5351 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:56.289107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a492d-6140-4744-8147-933ea3be5351 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:56.289383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a492d-6140-4744-8147-933ea3be5351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:56.335905 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f59a492d-6140-4744-8147-933ea3be5351 None None] MFA Rules not processed for user `f33ea26c1f72445bb4cb50ea503600dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:56.342512 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f59a492d-6140-4744-8147-933ea3be5351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:56.350170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b2ae8-7246-4636-8ab6-ad94476233e8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:56.363842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a3810-bb44-4413-8172-3fad6cbec175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:56.364708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a3810-bb44-4413-8172-3fad6cbec175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:56.365307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a3810-bb44-4413-8172-3fad6cbec175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:56.365737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a3810-bb44-4413-8172-3fad6cbec175 admin admin] PATH_INFO: `/v3/users/4684b684cb6c4c12818d1820b8995887` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:56.367507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a3810-bb44-4413-8172-3fad6cbec175 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4684b684cb6c4c12818d1820b8995887)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:56.370333 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a3810-bb44-4413-8172-3fad6cbec175 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:56.526308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-373a3810-bb44-4413-8172-3fad6cbec175 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4684b684cb6c4c12818d1820b8995887'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:56.529597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-373a3810-bb44-4413-8172-3fad6cbec175 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4684b684cb6c4c12818d1820b8995887'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:56.558311 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-373a3810-bb44-4413-8172-3fad6cbec175 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4684b684cb6c4c12818d1820b8995887'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:56.559465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-373a3810-bb44-4413-8172-3fad6cbec175 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4684b684cb6c4c12818d1820b8995887'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:56.590624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-373a3810-bb44-4413-8172-3fad6cbec175 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4684b684cb6c4c12818d1820b8995887'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:56.594010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 899/1804] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:56.595595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a492d-6140-4744-8147-933ea3be5351 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:56.610239 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f59a492d-6140-4744-8147-933ea3be5351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:56.628659 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 906/1805] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:14:56 2022] DELETE /identity/v3/users/4684b684cb6c4c12818d1820b8995887 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:56.735958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a492d-6140-4744-8147-933ea3be5351 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:56.788460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:57.100415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611c1d3-a5bd-4010-9ab4-b495bc17b8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:57.102378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611c1d3-a5bd-4010-9ab4-b495bc17b8d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:57.102893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611c1d3-a5bd-4010-9ab4-b495bc17b8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:57.103364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611c1d3-a5bd-4010-9ab4-b495bc17b8d6 admin admin] PATH_INFO: `/v3/projects/cd43c6d8d7a24c21816ea433bcd40af4/users/22894c7959aa4f6987ba77a7d9541434/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:57.312705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611c1d3-a5bd-4010-9ab4-b495bc17b8d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd43c6d8d7a24c21816ea433bcd40af4, user_id=22894c7959aa4f6987ba77a7d9541434, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:57.342845 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611c1d3-a5bd-4010-9ab4-b495bc17b8d6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:57.449524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 900/1806] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:56 2022] PUT /identity/v3/projects/cd43c6d8d7a24c21816ea433bcd40af4/users/22894c7959aa4f6987ba77a7d9541434/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 855 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:57.462781 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b21263-d7eb-4465-9474-9ede990dfff9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:57.463437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b21263-d7eb-4465-9474-9ede990dfff9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:57.463437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b21263-d7eb-4465-9474-9ede990dfff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:57.583001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c0b21263-d7eb-4465-9474-9ede990dfff9 None None] MFA Rules not processed for user `22894c7959aa4f6987ba77a7d9541434`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:57.588255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c0b21263-d7eb-4465-9474-9ede990dfff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:57.850056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611c1d3-a5bd-4010-9ab4-b495bc17b8d6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:57.850653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 907/1807] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:57.852671 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e611c1d3-a5bd-4010-9ab4-b495bc17b8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:57.994039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611c1d3-a5bd-4010-9ab4-b495bc17b8d6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:58.001672 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb02e11-475c-487f-a9ce-ea6b85d9ad79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:58.002366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb02e11-475c-487f-a9ce-ea6b85d9ad79 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.002642 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb02e11-475c-487f-a9ce-ea6b85d9ad79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.003178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb02e11-475c-487f-a9ce-ea6b85d9ad79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.003916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb02e11-475c-487f-a9ce-ea6b85d9ad79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:58.007090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb02e11-475c-487f-a9ce-ea6b85d9ad79 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:58.056373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 901/1808] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:57 2022] POST /identity/v3/projects => generated 368 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:14:58.060912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b21263-d7eb-4465-9474-9ede990dfff9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:58.079788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b21263-d7eb-4465-9474-9ede990dfff9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:58.140593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:58.247466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b05bc-6ed1-4b2e-bf90-9e0d90f6c6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:58.248160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b05bc-6ed1-4b2e-bf90-9e0d90f6c6ef admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.250944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b05bc-6ed1-4b2e-bf90-9e0d90f6c6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.251270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b05bc-6ed1-4b2e-bf90-9e0d90f6c6ef admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.252013 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b05bc-6ed1-4b2e-bf90-9e0d90f6c6ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:58.266495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b05bc-6ed1-4b2e-bf90-9e0d90f6c6ef admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:58.322376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 908/1809] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:58 2022] POST /identity/v3/users => generated 291 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:58.323702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb02e11-475c-487f-a9ce-ea6b85d9ad79 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:58.364536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb02e11-475c-487f-a9ce-ea6b85d9ad79 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:58.387031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9411c-ee8d-42ea-93fe-79b641908eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:58.387828 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9411c-ee8d-42ea-93fe-79b641908eec admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.388224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9411c-ee8d-42ea-93fe-79b641908eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.388631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9411c-ee8d-42ea-93fe-79b641908eec admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.389458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9411c-ee8d-42ea-93fe-79b641908eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:58.393032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9411c-ee8d-42ea-93fe-79b641908eec admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:58.412653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b05bc-6ed1-4b2e-bf90-9e0d90f6c6ef admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:58.414766 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-071b05bc-6ed1-4b2e-bf90-9e0d90f6c6ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:58.459934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b05bc-6ed1-4b2e-bf90-9e0d90f6c6ef admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:58.460138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 902/1810] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:58 2022] GET /identity/v3/roles => generated 1427 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:58.461986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9411c-ee8d-42ea-93fe-79b641908eec admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:58.471545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17030c1f-1b3e-4856-8740-730fffb95edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:58.472402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17030c1f-1b3e-4856-8740-730fffb95edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.472782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17030c1f-1b3e-4856-8740-730fffb95edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.473170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17030c1f-1b3e-4856-8740-730fffb95edf admin admin] PATH_INFO: `/v3/projects/02a4c24481f548cf81ea695c94121720` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.475999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17030c1f-1b3e-4856-8740-730fffb95edf admin admin] RBAC: Authorizing `identity:delete_project(project_id=02a4c24481f548cf81ea695c94121720)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:58.478062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17030c1f-1b3e-4856-8740-730fffb95edf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:58.485583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9411c-ee8d-42ea-93fe-79b641908eec admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:58.508604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84249f7d-7e12-4c8c-a334-5a38b85ed76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:58.509474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84249f7d-7e12-4c8c-a334-5a38b85ed76c admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.510492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84249f7d-7e12-4c8c-a334-5a38b85ed76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.510898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84249f7d-7e12-4c8c-a334-5a38b85ed76c admin admin] PATH_INFO: `/v3/projects/2f312ce0b8ad493ba6dbf0149663df4b/users/60fffa7b83454468a2d6d09916c2147c/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.557999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84249f7d-7e12-4c8c-a334-5a38b85ed76c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f312ce0b8ad493ba6dbf0149663df4b, user_id=60fffa7b83454468a2d6d09916c2147c, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:58.561290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84249f7d-7e12-4c8c-a334-5a38b85ed76c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:58.599837 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 903/1811] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:58 2022] PUT /identity/v3/projects/2f312ce0b8ad493ba6dbf0149663df4b/users/60fffa7b83454468a2d6d09916c2147c/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:58.604061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84249f7d-7e12-4c8c-a334-5a38b85ed76c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:58.620165 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-17030c1f-1b3e-4856-8740-730fffb95edf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02a4c24481f548cf81ea695c94121720', 'request_id': 'req-17030c1f-1b3e-4856-8740-730fffb95edf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:58.640702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-17030c1f-1b3e-4856-8740-730fffb95edf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02a4c24481f548cf81ea695c94121720', 'request_id': 'req-17030c1f-1b3e-4856-8740-730fffb95edf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:58.655836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-17030c1f-1b3e-4856-8740-730fffb95edf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02a4c24481f548cf81ea695c94121720', 'request_id': 'req-17030c1f-1b3e-4856-8740-730fffb95edf'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:58.692323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-17030c1f-1b3e-4856-8740-730fffb95edf admin admin] The token cache is being invalidate because project 02a4c24481f548cf81ea695c94121720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:14:58.694194 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-17030c1f-1b3e-4856-8740-730fffb95edf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:58.696356 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-17030c1f-1b3e-4856-8740-730fffb95edf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:14:58.699773 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 909/1812] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:14:58 2022] DELETE /identity/v3/projects/02a4c24481f548cf81ea695c94121720 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:58.704729 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21857cc-4284-4f52-8281-8ac91870d739 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.706540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21857cc-4284-4f52-8281-8ac91870d739 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.707076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21857cc-4284-4f52-8281-8ac91870d739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.764223 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:58.767210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e21857cc-4284-4f52-8281-8ac91870d739 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:58.770486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e21857cc-4284-4f52-8281-8ac91870d739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:58.789321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84249f7d-7e12-4c8c-a334-5a38b85ed76c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:58.936766 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 910/1813] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:58.949367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761cdd3-6763-4a7e-9045-6026f749b679 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.950422 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761cdd3-6763-4a7e-9045-6026f749b679 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.950945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761cdd3-6763-4a7e-9045-6026f749b679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.973138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a16d6d-d67e-49ad-b328-cc904c1f88aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:58.973777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a16d6d-d67e-49ad-b328-cc904c1f88aa admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:58.974058 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a16d6d-d67e-49ad-b328-cc904c1f88aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:58.975061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a16d6d-d67e-49ad-b328-cc904c1f88aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:58.975851 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a16d6d-d67e-49ad-b328-cc904c1f88aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:58.983087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a16d6d-d67e-49ad-b328-cc904c1f88aa admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:59.000822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d761cdd3-6763-4a7e-9045-6026f749b679 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:59.006632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d761cdd3-6763-4a7e-9045-6026f749b679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:59.014424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 904/1814] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:58 2022] GET /identity/v3/roles => generated 1427 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:59.015509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a16d6d-d67e-49ad-b328-cc904c1f88aa admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:59.018303 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-38a16d6d-d67e-49ad-b328-cc904c1f88aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:59.034541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a16d6d-d67e-49ad-b328-cc904c1f88aa admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:59.050081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 911/1815] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:59.052331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761cdd3-6763-4a7e-9045-6026f749b679 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:59.052606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5e4bc-108b-49af-81b1-0b840309cb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:59.053970 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5e4bc-108b-49af-81b1-0b840309cb2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:59.054243 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5e4bc-108b-49af-81b1-0b840309cb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:59.054527 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5e4bc-108b-49af-81b1-0b840309cb2d admin admin] PATH_INFO: `/v3/projects/2f312ce0b8ad493ba6dbf0149663df4b/users/60fffa7b83454468a2d6d09916c2147c/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:59.062628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5e4bc-108b-49af-81b1-0b840309cb2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f312ce0b8ad493ba6dbf0149663df4b, user_id=60fffa7b83454468a2d6d09916c2147c, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:59.068486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5e4bc-108b-49af-81b1-0b840309cb2d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:59.083119 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761cdd3-6763-4a7e-9045-6026f749b679 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:59.093891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 905/1816] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:59 2022] PUT /identity/v3/projects/2f312ce0b8ad493ba6dbf0149663df4b/users/60fffa7b83454468a2d6d09916c2147c/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:59.108671 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7764f7a-3a5a-482a-84cc-ece108a0e359 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:59.108958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7764f7a-3a5a-482a-84cc-ece108a0e359 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:59.109237 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7764f7a-3a5a-482a-84cc-ece108a0e359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:59.126920 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:59.152579 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e7764f7a-3a5a-482a-84cc-ece108a0e359 None None] MFA Rules not processed for user `60fffa7b83454468a2d6d09916c2147c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:14:59.157398 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e7764f7a-3a5a-482a-84cc-ece108a0e359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:59.192104 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee5902-4768-4910-9ef3-0aba70649c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:59.192980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5902-4768-4910-9ef3-0aba70649c73 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:59.193269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5902-4768-4910-9ef3-0aba70649c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:59.195008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee5902-4768-4910-9ef3-0aba70649c73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:59.195008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee5902-4768-4910-9ef3-0aba70649c73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:14:59.195008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee5902-4768-4910-9ef3-0aba70649c73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:59.196306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee5902-4768-4910-9ef3-0aba70649c73 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:59.212060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 912/1817] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:14:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:14:59.216933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee5902-4768-4910-9ef3-0aba70649c73 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:59.255552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee5902-4768-4910-9ef3-0aba70649c73 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:59.279903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e72d26-08f2-4737-9ff5-45cd4dc39b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:59.281514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e72d26-08f2-4737-9ff5-45cd4dc39b04 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:59.283093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e72d26-08f2-4737-9ff5-45cd4dc39b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:59.283487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e72d26-08f2-4737-9ff5-45cd4dc39b04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:59.284598 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e72d26-08f2-4737-9ff5-45cd4dc39b04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:59.292598 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e72d26-08f2-4737-9ff5-45cd4dc39b04 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:59.297793 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 906/1818] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:14:59.302601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7764f7a-3a5a-482a-84cc-ece108a0e359 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:59.310501 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e7764f7a-3a5a-482a-84cc-ece108a0e359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:59.379803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 913/1819] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:14:59 2022] POST /identity/v3/projects => generated 408 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:14:59.386840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e72d26-08f2-4737-9ff5-45cd4dc39b04 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:59.389066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7764f7a-3a5a-482a-84cc-ece108a0e359 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:59.437947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:14:59.510864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b47a52-8c76-4d2f-8ce9-4aa2deac9d12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:59.511681 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b47a52-8c76-4d2f-8ce9-4aa2deac9d12 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:59.512119 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b47a52-8c76-4d2f-8ce9-4aa2deac9d12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:59.512400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b47a52-8c76-4d2f-8ce9-4aa2deac9d12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:59.576973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e72d26-08f2-4737-9ff5-45cd4dc39b04 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:59.596915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584e6de-df8d-4e26-ab1e-a04ebc56229c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:59.597793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584e6de-df8d-4e26-ab1e-a04ebc56229c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:59.598220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584e6de-df8d-4e26-ab1e-a04ebc56229c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:59.598638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a584e6de-df8d-4e26-ab1e-a04ebc56229c admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:59.600289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584e6de-df8d-4e26-ab1e-a04ebc56229c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:59.602442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a584e6de-df8d-4e26-ab1e-a04ebc56229c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:59.638125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b47a52-8c76-4d2f-8ce9-4aa2deac9d12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:59.639888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b47a52-8c76-4d2f-8ce9-4aa2deac9d12 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:59.642157 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 914/1820] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:14:59 2022] POST /identity/v3/users => generated 305 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:14:59.646409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584e6de-df8d-4e26-ab1e-a04ebc56229c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:59.657631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 907/1821] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:14:59.660857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a584e6de-df8d-4e26-ab1e-a04ebc56229c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:59.668438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cbeec-2990-4f21-9207-697d7e3c35d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:59.669292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cbeec-2990-4f21-9207-697d7e3c35d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:59.669594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cbeec-2990-4f21-9207-697d7e3c35d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:59.669935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cbeec-2990-4f21-9207-697d7e3c35d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:59.670909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35cbeec-2990-4f21-9207-697d7e3c35d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:59.672709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35cbeec-2990-4f21-9207-697d7e3c35d1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:59.690165 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 915/1822] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:59 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:14:59.694202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b47a52-8c76-4d2f-8ce9-4aa2deac9d12 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:59.716453 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b47a52-8c76-4d2f-8ce9-4aa2deac9d12 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:59.742883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cbeec-2990-4f21-9207-697d7e3c35d1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:59.745438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c35cbeec-2990-4f21-9207-697d7e3c35d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:59.769235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3b08f-e95f-47ff-bb0e-a137af125525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:59.769881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3b08f-e95f-47ff-bb0e-a137af125525 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:59.771575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cbeec-2990-4f21-9207-697d7e3c35d1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:59.772550 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3b08f-e95f-47ff-bb0e-a137af125525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:59.772878 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3b08f-e95f-47ff-bb0e-a137af125525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:59.773925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3b08f-e95f-47ff-bb0e-a137af125525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:59.786693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53686d-abc8-434e-84d5-162dc0946ed3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:59.787479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53686d-abc8-434e-84d5-162dc0946ed3 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:59.787872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53686d-abc8-434e-84d5-162dc0946ed3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:59.788436 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53686d-abc8-434e-84d5-162dc0946ed3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:59.789694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3b08f-e95f-47ff-bb0e-a137af125525 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:59.791966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bf53686d-abc8-434e-84d5-162dc0946ed3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:14:59.814424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 908/1823] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:14:59 2022] GET /identity/v3/roles => generated 1427 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:14:59.826685 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3b08f-e95f-47ff-bb0e-a137af125525 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:14:59.841467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3b08f-e95f-47ff-bb0e-a137af125525 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:14:59.862707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237eb76-21d8-4d8d-a674-18e7a2d673c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:14:59.863282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237eb76-21d8-4d8d-a674-18e7a2d673c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:59.863561 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237eb76-21d8-4d8d-a674-18e7a2d673c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:59.863841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237eb76-21d8-4d8d-a674-18e7a2d673c1 admin admin] PATH_INFO: `/v3/projects/99ca28da9fbb4a0499c1308445d03419/users/18b5c2efef694a5cbbd2ec60f7ad30aa/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:14:59.924832 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237eb76-21d8-4d8d-a674-18e7a2d673c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ca28da9fbb4a0499c1308445d03419, user_id=18b5c2efef694a5cbbd2ec60f7ad30aa, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:59.932364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237eb76-21d8-4d8d-a674-18e7a2d673c1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:59.970149 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 909/1824] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:14:59 2022] PUT /identity/v3/projects/99ca28da9fbb4a0499c1308445d03419/users/18b5c2efef694a5cbbd2ec60f7ad30aa/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:14:59.971899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53686d-abc8-434e-84d5-162dc0946ed3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:14:59.974035 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53686d-abc8-434e-84d5-162dc0946ed3 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:14:59.980114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd0c62-2ebc-4250-a406-d93c198e4981 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:14:59.980547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd0c62-2ebc-4250-a406-d93c198e4981 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:14:59.981061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd0c62-2ebc-4250-a406-d93c198e4981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:00.004181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 916/1825] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:14:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:00.060917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a6fd0c62-2ebc-4250-a406-d93c198e4981 None None] MFA Rules not processed for user `18b5c2efef694a5cbbd2ec60f7ad30aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:00.062331 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a6fd0c62-2ebc-4250-a406-d93c198e4981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:00.184726 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 910/1826] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:14:59 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:00.191252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53686d-abc8-434e-84d5-162dc0946ed3 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:00.218008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53686d-abc8-434e-84d5-162dc0946ed3 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:00.240152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:00.291334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6c21c-8f2b-4c1c-b7eb-3897b647b3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:00.292275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6c21c-8f2b-4c1c-b7eb-3897b647b3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:00.292680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6c21c-8f2b-4c1c-b7eb-3897b647b3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:00.293023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6c21c-8f2b-4c1c-b7eb-3897b647b3b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:00.293775 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6c21c-8f2b-4c1c-b7eb-3897b647b3b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:00.295796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6c21c-8f2b-4c1c-b7eb-3897b647b3b6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:00.330856 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 917/1827] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:00 2022] POST /identity/v3/projects => generated 408 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:00.335884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd0c62-2ebc-4250-a406-d93c198e4981 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:00.348324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd0c62-2ebc-4250-a406-d93c198e4981 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:00.376218 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:00.415718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42186247-e9f9-45ef-b31c-320500378c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:00.416567 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42186247-e9f9-45ef-b31c-320500378c87 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:00.416854 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42186247-e9f9-45ef-b31c-320500378c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:00.417131 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42186247-e9f9-45ef-b31c-320500378c87 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:00.418143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42186247-e9f9-45ef-b31c-320500378c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:00.420580 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42186247-e9f9-45ef-b31c-320500378c87 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:00.472960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 911/1828] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:00 2022] POST /identity/v3/users => generated 311 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:00.477829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6c21c-8f2b-4c1c-b7eb-3897b647b3b6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:00.492693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6c21c-8f2b-4c1c-b7eb-3897b647b3b6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:00.500984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c4e66-d53d-420e-951b-109d7d381ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:00.502522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c4e66-d53d-420e-951b-109d7d381ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:00.503099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c4e66-d53d-420e-951b-109d7d381ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:00.504074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c4e66-d53d-420e-951b-109d7d381ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:00.504500 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c4e66-d53d-420e-951b-109d7d381ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:00.506289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c4e66-d53d-420e-951b-109d7d381ab5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:00.525642 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 918/1829] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:00 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:00.530293 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42186247-e9f9-45ef-b31c-320500378c87 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:00.552470 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42186247-e9f9-45ef-b31c-320500378c87 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:00.560547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e97c-26ca-4e08-a3ec-b1c6676718fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:00.561355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e97c-26ca-4e08-a3ec-b1c6676718fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:00.561577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e97c-26ca-4e08-a3ec-b1c6676718fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:00.561865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928e97c-26ca-4e08-a3ec-b1c6676718fa admin admin] PATH_INFO: `/v3/projects/240170495f714ffaaf4389d4471023b4/users/25237b6239504d3fa20d7c02321cec55/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:00.585172 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928e97c-26ca-4e08-a3ec-b1c6676718fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=240170495f714ffaaf4389d4471023b4, user_id=25237b6239504d3fa20d7c02321cec55, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:00.589854 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928e97c-26ca-4e08-a3ec-b1c6676718fa admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:00.615108 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 912/1830] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:00 2022] PUT /identity/v3/projects/240170495f714ffaaf4389d4471023b4/users/25237b6239504d3fa20d7c02321cec55/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:00.618768 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c4e66-d53d-420e-951b-109d7d381ab5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:00.638284 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c4e66-d53d-420e-951b-109d7d381ab5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:00.722970 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c8a0dc-687e-499b-bd9a-43361c712b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:00.723599 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8a0dc-687e-499b-bd9a-43361c712b76 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:00.723886 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8a0dc-687e-499b-bd9a-43361c712b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:00.724165 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c8a0dc-687e-499b-bd9a-43361c712b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:00.724909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8a0dc-687e-499b-bd9a-43361c712b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:00.726528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c8a0dc-687e-499b-bd9a-43361c712b76 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:00.742450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 919/1831] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:00 2022] GET /identity/v3/roles => generated 1427 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:00.746827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e97c-26ca-4e08-a3ec-b1c6676718fa admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:00.758510 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928e97c-26ca-4e08-a3ec-b1c6676718fa admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:00.764264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe8b31-6199-4426-bdaa-fdef46a97fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:00.764846 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe8b31-6199-4426-bdaa-fdef46a97fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:00.765344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe8b31-6199-4426-bdaa-fdef46a97fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:00.765711 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe8b31-6199-4426-bdaa-fdef46a97fd5 admin admin] PATH_INFO: `/v3/projects/240170495f714ffaaf4389d4471023b4/users/25237b6239504d3fa20d7c02321cec55/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:00.768272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe8b31-6199-4426-bdaa-fdef46a97fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=240170495f714ffaaf4389d4471023b4, user_id=25237b6239504d3fa20d7c02321cec55, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:00.771617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe8b31-6199-4426-bdaa-fdef46a97fd5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:00.784377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 913/1832] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:00 2022] PUT /identity/v3/projects/240170495f714ffaaf4389d4471023b4/users/25237b6239504d3fa20d7c02321cec55/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:00.796967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99272a6-4394-48ab-8a88-95ce1f597817 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:00.797258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99272a6-4394-48ab-8a88-95ce1f597817 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:00.797537 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99272a6-4394-48ab-8a88-95ce1f597817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:00.831096 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d99272a6-4394-48ab-8a88-95ce1f597817 None None] MFA Rules not processed for user `25237b6239504d3fa20d7c02321cec55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:00.832577 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d99272a6-4394-48ab-8a88-95ce1f597817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:00.927670 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 920/1833] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:00 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:00.936226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe8b31-6199-4426-bdaa-fdef46a97fd5 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:01.037395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe8b31-6199-4426-bdaa-fdef46a97fd5 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:01.043808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33952f5e-397a-4216-9c5c-6ed20df2c20c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:01.044661 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33952f5e-397a-4216-9c5c-6ed20df2c20c service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:01.044948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33952f5e-397a-4216-9c5c-6ed20df2c20c service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:01.045219 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33952f5e-397a-4216-9c5c-6ed20df2c20c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:01.062695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33952f5e-397a-4216-9c5c-6ed20df2c20c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:01.062695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33952f5e-397a-4216-9c5c-6ed20df2c20c service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:01.078258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 914/1834] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:00 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:01.414402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99272a6-4394-48ab-8a88-95ce1f597817 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:01.435096 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99272a6-4394-48ab-8a88-95ce1f597817 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:01.439573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20befd60-2eb2-4210-9af3-10df0e70d8ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:01.440221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20befd60-2eb2-4210-9af3-10df0e70d8ef service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:01.440512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20befd60-2eb2-4210-9af3-10df0e70d8ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:01.440785 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20befd60-2eb2-4210-9af3-10df0e70d8ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:01.460874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:01.511178 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20befd60-2eb2-4210-9af3-10df0e70d8ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:01.512611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20befd60-2eb2-4210-9af3-10df0e70d8ef service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:01.521566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 921/1835] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:01 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:03.656584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33952f5e-397a-4216-9c5c-6ed20df2c20c service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:03.744130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33952f5e-397a-4216-9c5c-6ed20df2c20c service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:03.750858 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a9dd1-98a1-4ac5-8a27-3636cdd096cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:03.751463 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a9dd1-98a1-4ac5-8a27-3636cdd096cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:03.751876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a9dd1-98a1-4ac5-8a27-3636cdd096cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:03.755072 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a9dd1-98a1-4ac5-8a27-3636cdd096cd admin admin] PATH_INFO: `/v3/users/22894c7959aa4f6987ba77a7d9541434` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:03.756637 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a9dd1-98a1-4ac5-8a27-3636cdd096cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=22894c7959aa4f6987ba77a7d9541434)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:03.758845 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a9dd1-98a1-4ac5-8a27-3636cdd096cd admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:03.784052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:03.821277 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-020a9dd1-98a1-4ac5-8a27-3636cdd096cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22894c7959aa4f6987ba77a7d9541434'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:03.833765 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-020a9dd1-98a1-4ac5-8a27-3636cdd096cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22894c7959aa4f6987ba77a7d9541434'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:03.834860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-020a9dd1-98a1-4ac5-8a27-3636cdd096cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22894c7959aa4f6987ba77a7d9541434'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:03.835564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-020a9dd1-98a1-4ac5-8a27-3636cdd096cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22894c7959aa4f6987ba77a7d9541434'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:03.852539 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-020a9dd1-98a1-4ac5-8a27-3636cdd096cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22894c7959aa4f6987ba77a7d9541434'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:03.866959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 915/1836] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:03 2022] DELETE /identity/v3/users/22894c7959aa4f6987ba77a7d9541434 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:03.871469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddec6a-ed2d-41f2-9b62-38d0a183ccfb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:03.871971 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddec6a-ed2d-41f2-9b62-38d0a183ccfb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:03.872058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddec6a-ed2d-41f2-9b62-38d0a183ccfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:03.891579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e3ddec6a-ed2d-41f2-9b62-38d0a183ccfb None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:03.892798 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e3ddec6a-ed2d-41f2-9b62-38d0a183ccfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:03.952638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 922/1837] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:03.965206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a9dd1-98a1-4ac5-8a27-3636cdd096cd admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:03.965206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-020a9dd1-98a1-4ac5-8a27-3636cdd096cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:04.021447 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a9dd1-98a1-4ac5-8a27-3636cdd096cd admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:04.025084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a537e2-99ca-4190-ba07-8e766d1716d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:04.026273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a537e2-99ca-4190-ba07-8e766d1716d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:04.026556 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a537e2-99ca-4190-ba07-8e766d1716d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:04.026872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a537e2-99ca-4190-ba07-8e766d1716d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:04.036126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a537e2-99ca-4190-ba07-8e766d1716d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:04.037575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a537e2-99ca-4190-ba07-8e766d1716d7 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:04.047442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 916/1838] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:04.331091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddec6a-ed2d-41f2-9b62-38d0a183ccfb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:04.334102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e3ddec6a-ed2d-41f2-9b62-38d0a183ccfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:04.358037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddec6a-ed2d-41f2-9b62-38d0a183ccfb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:04.363377 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:04.364047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:04.364332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:04.364645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] PATH_INFO: `/v3/projects/cd43c6d8d7a24c21816ea433bcd40af4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:04.365943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd43c6d8d7a24c21816ea433bcd40af4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:04.367944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:04.422191 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd43c6d8d7a24c21816ea433bcd40af4', 'request_id': 'req-bf301856-6728-478e-9ccc-48c188f5512a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:04.428839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd43c6d8d7a24c21816ea433bcd40af4', 'request_id': 'req-bf301856-6728-478e-9ccc-48c188f5512a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:04.435878 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd43c6d8d7a24c21816ea433bcd40af4', 'request_id': 'req-bf301856-6728-478e-9ccc-48c188f5512a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:04.450085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] The token cache is being invalidate because project cd43c6d8d7a24c21816ea433bcd40af4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:04.450384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:04.451169 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:04.452756 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 923/1839] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:04 2022] DELETE /identity/v3/projects/cd43c6d8d7a24c21816ea433bcd40af4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:04.456818 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a537e2-99ca-4190-ba07-8e766d1716d7 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:04.466896 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a537e2-99ca-4190-ba07-8e766d1716d7 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:04.549932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98510c6-cef7-4008-8ffc-37e7bbedadc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:04.551029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98510c6-cef7-4008-8ffc-37e7bbedadc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:04.551482 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98510c6-cef7-4008-8ffc-37e7bbedadc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:04.551741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98510c6-cef7-4008-8ffc-37e7bbedadc0 admin admin] PATH_INFO: `/v3/users/60fffa7b83454468a2d6d09916c2147c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:04.553048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98510c6-cef7-4008-8ffc-37e7bbedadc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60fffa7b83454468a2d6d09916c2147c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:04.555690 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98510c6-cef7-4008-8ffc-37e7bbedadc0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:04.629143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:04.631197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f98510c6-cef7-4008-8ffc-37e7bbedadc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60fffa7b83454468a2d6d09916c2147c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:04.641584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f98510c6-cef7-4008-8ffc-37e7bbedadc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60fffa7b83454468a2d6d09916c2147c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:04.642547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f98510c6-cef7-4008-8ffc-37e7bbedadc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60fffa7b83454468a2d6d09916c2147c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:04.643415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f98510c6-cef7-4008-8ffc-37e7bbedadc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60fffa7b83454468a2d6d09916c2147c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:04.657873 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f98510c6-cef7-4008-8ffc-37e7bbedadc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60fffa7b83454468a2d6d09916c2147c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:04.681330 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 917/1840] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:04 2022] DELETE /identity/v3/users/60fffa7b83454468a2d6d09916c2147c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:04.990391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:04.992585 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:05.007994 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf301856-6728-478e-9ccc-48c188f5512a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:05.068402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:05.111151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:05.111841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:05.112129 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:05.112408 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc admin admin] PATH_INFO: `/v3/projects/2f312ce0b8ad493ba6dbf0149663df4b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:05.113623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f312ce0b8ad493ba6dbf0149663df4b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:05.115685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:05.208619 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f312ce0b8ad493ba6dbf0149663df4b', 'request_id': 'req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:05.219815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f312ce0b8ad493ba6dbf0149663df4b', 'request_id': 'req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:05.226330 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f312ce0b8ad493ba6dbf0149663df4b', 'request_id': 'req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:05.252935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc admin admin] The token cache is being invalidate because project 2f312ce0b8ad493ba6dbf0149663df4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:05.253261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:05.253956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2dd5f93f-2e8d-40e0-aa8f-1202fee7badc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:05.255969 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 924/1841] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:04 2022] DELETE /identity/v3/projects/2f312ce0b8ad493ba6dbf0149663df4b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:05.261584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee741c18-5381-45db-8ede-01588606d165 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:05.261804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee741c18-5381-45db-8ede-01588606d165 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:05.262112 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee741c18-5381-45db-8ede-01588606d165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:05.299404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ee741c18-5381-45db-8ede-01588606d165 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:05.300747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ee741c18-5381-45db-8ede-01588606d165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:05.378251 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 918/1842] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:05.387854 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b9e88-629b-4a1e-a8a0-1d2eba03ab32 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:05.388357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b9e88-629b-4a1e-a8a0-1d2eba03ab32 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:05.388775 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b9e88-629b-4a1e-a8a0-1d2eba03ab32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:05.418815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-fa1b9e88-629b-4a1e-a8a0-1d2eba03ab32 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:05.420367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fa1b9e88-629b-4a1e-a8a0-1d2eba03ab32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:05.428701 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 925/1843] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:05.433119 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee741c18-5381-45db-8ede-01588606d165 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:05.445984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee741c18-5381-45db-8ede-01588606d165 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:05.451653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1aa7a1-09b2-42a8-94b3-4358c27ea1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:05.452262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1aa7a1-09b2-42a8-94b3-4358c27ea1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:05.452551 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1aa7a1-09b2-42a8-94b3-4358c27ea1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:05.452831 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1aa7a1-09b2-42a8-94b3-4358c27ea1b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:05.453538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1aa7a1-09b2-42a8-94b3-4358c27ea1b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:05.453850 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1aa7a1-09b2-42a8-94b3-4358c27ea1b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:05.455643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1aa7a1-09b2-42a8-94b3-4358c27ea1b3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:05.481693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 919/1844] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:15:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:15:05.482051 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b9e88-629b-4a1e-a8a0-1d2eba03ab32 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:05.487149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b9e88-629b-4a1e-a8a0-1d2eba03ab32 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:05.493255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1d3b4-0b6c-4a8c-95ba-12b25dd790be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:05.494099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1d3b4-0b6c-4a8c-95ba-12b25dd790be admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:05.494750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1d3b4-0b6c-4a8c-95ba-12b25dd790be admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:05.495174 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1d3b4-0b6c-4a8c-95ba-12b25dd790be admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:05.496472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1d3b4-0b6c-4a8c-95ba-12b25dd790be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:05.498632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1d3b4-0b6c-4a8c-95ba-12b25dd790be admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:05.560044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 926/1845] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:05 2022] POST /identity/v3/projects => generated 386 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:05.561551 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1aa7a1-09b2-42a8-94b3-4358c27ea1b3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:05.574395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1aa7a1-09b2-42a8-94b3-4358c27ea1b3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:05.664779 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0f4d0-9d07-401f-845f-769ba0f96f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:05.665545 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0f4d0-9d07-401f-845f-769ba0f96f8a admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:05.665962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0f4d0-9d07-401f-845f-769ba0f96f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:05.666329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0f4d0-9d07-401f-845f-769ba0f96f8a admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:05.667199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0f4d0-9d07-401f-845f-769ba0f96f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:05.669397 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0f4d0-9d07-401f-845f-769ba0f96f8a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:05.721402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 920/1846] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:05 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:05.725818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1d3b4-0b6c-4a8c-95ba-12b25dd790be admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:05.749492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1d3b4-0b6c-4a8c-95ba-12b25dd790be admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:05.773147 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d80fd-b671-45dd-a553-b2e6d165fe95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:05.773998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d80fd-b671-45dd-a553-b2e6d165fe95 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:05.774387 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d80fd-b671-45dd-a553-b2e6d165fe95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:05.774991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d80fd-b671-45dd-a553-b2e6d165fe95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:05.775991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d80fd-b671-45dd-a553-b2e6d165fe95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:05.779537 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d80fd-b671-45dd-a553-b2e6d165fe95 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:05.793355 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 927/1847] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:05 2022] GET /identity/v3/roles => generated 1427 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:05.797293 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0f4d0-9d07-401f-845f-769ba0f96f8a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:05.810861 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0f4d0-9d07-401f-845f-769ba0f96f8a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:05.827458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87af95-57c1-4db8-b7dd-4cec3e148dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:05.828209 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87af95-57c1-4db8-b7dd-4cec3e148dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:05.828439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87af95-57c1-4db8-b7dd-4cec3e148dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:05.828717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87af95-57c1-4db8-b7dd-4cec3e148dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:05.830062 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87af95-57c1-4db8-b7dd-4cec3e148dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:05.834890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87af95-57c1-4db8-b7dd-4cec3e148dd9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:05.855493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 921/1848] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:05 2022] GET /identity/v3/roles => generated 1427 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:05.859799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d80fd-b671-45dd-a553-b2e6d165fe95 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:05.870727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d80fd-b671-45dd-a553-b2e6d165fe95 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:05.875560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f50f8-ced5-4370-b755-b676cf3876ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:05.876616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f50f8-ced5-4370-b755-b676cf3876ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:05.876902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f50f8-ced5-4370-b755-b676cf3876ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:05.877347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f50f8-ced5-4370-b755-b676cf3876ca admin admin] PATH_INFO: `/v3/projects/042de36fb7d541c79c10c965923af0d3/users/28345906e41c4cc690217b5fff147fa6/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:05.912937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f50f8-ced5-4370-b755-b676cf3876ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=042de36fb7d541c79c10c965923af0d3, user_id=28345906e41c4cc690217b5fff147fa6, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:05.916404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f50f8-ced5-4370-b755-b676cf3876ca admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:05.936141 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 928/1849] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:05 2022] PUT /identity/v3/projects/042de36fb7d541c79c10c965923af0d3/users/28345906e41c4cc690217b5fff147fa6/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:05.946733 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc7385-b75d-42db-add5-58b480790f7d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:05.947991 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc7385-b75d-42db-add5-58b480790f7d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:05.948272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc7385-b75d-42db-add5-58b480790f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:05.982246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4cdc7385-b75d-42db-add5-58b480790f7d None None] MFA Rules not processed for user `28345906e41c4cc690217b5fff147fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:05.983536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4cdc7385-b75d-42db-add5-58b480790f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:06.008199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:06.064432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 922/1850] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:06.068937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f50f8-ced5-4370-b755-b676cf3876ca admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:06.158184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f50f8-ced5-4370-b755-b676cf3876ca admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:06.167246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a8a24-1190-4fc1-b1ea-bb7060dbc2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:06.167861 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a8a24-1190-4fc1-b1ea-bb7060dbc2bf admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:06.168146 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a8a24-1190-4fc1-b1ea-bb7060dbc2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:06.168426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a8a24-1190-4fc1-b1ea-bb7060dbc2bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:06.169107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a8a24-1190-4fc1-b1ea-bb7060dbc2bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:06.171591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a8a24-1190-4fc1-b1ea-bb7060dbc2bf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:06.191687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:06.231378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 929/1851] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:06 2022] POST /identity/v3/projects => generated 384 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:06.236816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc7385-b75d-42db-add5-58b480790f7d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:06.288734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc7385-b75d-42db-add5-58b480790f7d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:06.441574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4288ad-9bff-419a-bd5b-74d21f6cdf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:06.442372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4288ad-9bff-419a-bd5b-74d21f6cdf00 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:06.442708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4288ad-9bff-419a-bd5b-74d21f6cdf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:06.443610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4288ad-9bff-419a-bd5b-74d21f6cdf00 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:06.444327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4288ad-9bff-419a-bd5b-74d21f6cdf00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:06.446577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4288ad-9bff-419a-bd5b-74d21f6cdf00 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:06.470571 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 923/1852] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:06 2022] POST /identity/v3/users => generated 299 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:06.475762 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a8a24-1190-4fc1-b1ea-bb7060dbc2bf admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:06.486577 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a8a24-1190-4fc1-b1ea-bb7060dbc2bf admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:06.494289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee05d5-2f38-44f2-a80d-a40d395b924b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:06.495210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee05d5-2f38-44f2-a80d-a40d395b924b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:06.495500 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee05d5-2f38-44f2-a80d-a40d395b924b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:06.495780 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee05d5-2f38-44f2-a80d-a40d395b924b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:06.496560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee05d5-2f38-44f2-a80d-a40d395b924b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:06.498335 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee05d5-2f38-44f2-a80d-a40d395b924b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:06.517109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 930/1853] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:06 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:06.521331 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4288ad-9bff-419a-bd5b-74d21f6cdf00 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:06.535567 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4288ad-9bff-419a-bd5b-74d21f6cdf00 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:06.542288 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6d272-bb3a-41ba-a24a-9ce8730f2799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:06.543016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6d272-bb3a-41ba-a24a-9ce8730f2799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:06.543305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6d272-bb3a-41ba-a24a-9ce8730f2799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:06.543584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6d272-bb3a-41ba-a24a-9ce8730f2799 admin admin] PATH_INFO: `/v3/projects/a9772b32fd494e89be133523ab9bcbd3/users/a62969071cd944b0825189f4520ad3e1/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:06.560215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a6d272-bb3a-41ba-a24a-9ce8730f2799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9772b32fd494e89be133523ab9bcbd3, user_id=a62969071cd944b0825189f4520ad3e1, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:06.562934 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a6d272-bb3a-41ba-a24a-9ce8730f2799 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:06.575655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 924/1854] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:06 2022] PUT /identity/v3/projects/a9772b32fd494e89be133523ab9bcbd3/users/a62969071cd944b0825189f4520ad3e1/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:06.579615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee05d5-2f38-44f2-a80d-a40d395b924b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:06.591369 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee05d5-2f38-44f2-a80d-a40d395b924b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:06.662275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232a27f-b0ec-43fd-81cc-619b67a094c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:06.663245 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232a27f-b0ec-43fd-81cc-619b67a094c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:06.663528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232a27f-b0ec-43fd-81cc-619b67a094c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:06.663793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232a27f-b0ec-43fd-81cc-619b67a094c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:06.664502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3232a27f-b0ec-43fd-81cc-619b67a094c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:06.666947 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3232a27f-b0ec-43fd-81cc-619b67a094c8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:06.682189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 931/1855] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:06 2022] GET /identity/v3/roles => generated 1427 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:06.683441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6d272-bb3a-41ba-a24a-9ce8730f2799 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:06.694546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6d272-bb3a-41ba-a24a-9ce8730f2799 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:06.699445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3296b0-f5b8-45db-8f82-e595e9c5ebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:06.700137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3296b0-f5b8-45db-8f82-e595e9c5ebfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:06.700418 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3296b0-f5b8-45db-8f82-e595e9c5ebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:06.700795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3296b0-f5b8-45db-8f82-e595e9c5ebfc admin admin] PATH_INFO: `/v3/projects/a9772b32fd494e89be133523ab9bcbd3/users/a62969071cd944b0825189f4520ad3e1/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:06.702906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3296b0-f5b8-45db-8f82-e595e9c5ebfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9772b32fd494e89be133523ab9bcbd3, user_id=a62969071cd944b0825189f4520ad3e1, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:06.705941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3296b0-f5b8-45db-8f82-e595e9c5ebfc admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:06.717681 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 925/1856] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:06 2022] PUT /identity/v3/projects/a9772b32fd494e89be133523ab9bcbd3/users/a62969071cd944b0825189f4520ad3e1/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:06.732362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de9d6-a33a-42c5-bb68-71ead7c22386 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:06.732650 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de9d6-a33a-42c5-bb68-71ead7c22386 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:06.732933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4de9d6-a33a-42c5-bb68-71ead7c22386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:06.775750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ed4de9d6-a33a-42c5-bb68-71ead7c22386 None None] MFA Rules not processed for user `a62969071cd944b0825189f4520ad3e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:06.777015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ed4de9d6-a33a-42c5-bb68-71ead7c22386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:06.879323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 932/1857] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:06.896444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3296b0-f5b8-45db-8f82-e595e9c5ebfc admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:06.898444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4d3296b0-f5b8-45db-8f82-e595e9c5ebfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:06.980143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3296b0-f5b8-45db-8f82-e595e9c5ebfc admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:06.984085 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9b3e4-9af7-42a3-9f29-81e88ed7def6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:06.984948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9b3e4-9af7-42a3-9f29-81e88ed7def6 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:06.985260 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9b3e4-9af7-42a3-9f29-81e88ed7def6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:06.985520 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9b3e4-9af7-42a3-9f29-81e88ed7def6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:06.996102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9b3e4-9af7-42a3-9f29-81e88ed7def6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:06.997617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b9b3e4-9af7-42a3-9f29-81e88ed7def6 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:07.010752 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 926/1858] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:07.055277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4de9d6-a33a-42c5-bb68-71ead7c22386 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:07.170049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4de9d6-a33a-42c5-bb68-71ead7c22386 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:07.182173 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f6827-445d-4754-9dcd-7cdcded1b3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:07.183672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f6827-445d-4754-9dcd-7cdcded1b3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:07.183956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f6827-445d-4754-9dcd-7cdcded1b3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:07.184236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f6827-445d-4754-9dcd-7cdcded1b3aa admin admin] PATH_INFO: `/v3/users/28345906e41c4cc690217b5fff147fa6` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:07.185575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f6827-445d-4754-9dcd-7cdcded1b3aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=28345906e41c4cc690217b5fff147fa6)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:07.187623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f6827-445d-4754-9dcd-7cdcded1b3aa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:07.189829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:07.260539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d1f6827-445d-4754-9dcd-7cdcded1b3aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28345906e41c4cc690217b5fff147fa6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.260977 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d1f6827-445d-4754-9dcd-7cdcded1b3aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28345906e41c4cc690217b5fff147fa6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.272728 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d1f6827-445d-4754-9dcd-7cdcded1b3aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28345906e41c4cc690217b5fff147fa6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.273478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d1f6827-445d-4754-9dcd-7cdcded1b3aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28345906e41c4cc690217b5fff147fa6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.286919 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d1f6827-445d-4754-9dcd-7cdcded1b3aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28345906e41c4cc690217b5fff147fa6'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:07.301052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 933/1859] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:07 2022] DELETE /identity/v3/users/28345906e41c4cc690217b5fff147fa6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:07.305638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067ca35-a553-4f48-aa4c-9a043f8a8e8a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:07.305926 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067ca35-a553-4f48-aa4c-9a043f8a8e8a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:07.306205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067ca35-a553-4f48-aa4c-9a043f8a8e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:07.326936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5067ca35-a553-4f48-aa4c-9a043f8a8e8a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:07.328269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5067ca35-a553-4f48-aa4c-9a043f8a8e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:07.341660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:07.401235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 927/1860] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:07.410911 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f6827-445d-4754-9dcd-7cdcded1b3aa admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:07.412486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5d1f6827-445d-4754-9dcd-7cdcded1b3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:07.520370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f6827-445d-4754-9dcd-7cdcded1b3aa admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:07.524923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb20f6-f96a-447d-89eb-7ff6885f32ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:07.526342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb20f6-f96a-447d-89eb-7ff6885f32ee service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:07.526810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb20f6-f96a-447d-89eb-7ff6885f32ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:07.527172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb20f6-f96a-447d-89eb-7ff6885f32ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:07.551825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb20f6-f96a-447d-89eb-7ff6885f32ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:07.553627 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb20f6-f96a-447d-89eb-7ff6885f32ee service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:07.575874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 934/1861] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:07.987844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067ca35-a553-4f48-aa4c-9a043f8a8e8a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:07.990130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5067ca35-a553-4f48-aa4c-9a043f8a8e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:08.004857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067ca35-a553-4f48-aa4c-9a043f8a8e8a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:08.011066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:08.012008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:08.012477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:08.012999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] PATH_INFO: `/v3/projects/042de36fb7d541c79c10c965923af0d3` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:08.015061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] RBAC: Authorizing `identity:delete_project(project_id=042de36fb7d541c79c10c965923af0d3)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:08.017052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:08.073173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '042de36fb7d541c79c10c965923af0d3', 'request_id': 'req-1389c6b0-0652-4a87-9171-26218e66e60c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.079676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '042de36fb7d541c79c10c965923af0d3', 'request_id': 'req-1389c6b0-0652-4a87-9171-26218e66e60c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.090817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '042de36fb7d541c79c10c965923af0d3', 'request_id': 'req-1389c6b0-0652-4a87-9171-26218e66e60c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.104511 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] The token cache is being invalidate because project 042de36fb7d541c79c10c965923af0d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:08.104875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.105597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.107308 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 928/1862] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:07 2022] DELETE /identity/v3/projects/042de36fb7d541c79c10c965923af0d3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:08.111435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb20f6-f96a-447d-89eb-7ff6885f32ee service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:08.122395 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb20f6-f96a-447d-89eb-7ff6885f32ee service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:08.186563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141fb0b-6e31-474c-8796-09f1b6b9477f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:08.187315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141fb0b-6e31-474c-8796-09f1b6b9477f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:08.187441 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141fb0b-6e31-474c-8796-09f1b6b9477f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:08.187727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141fb0b-6e31-474c-8796-09f1b6b9477f admin admin] PATH_INFO: `/v3/users/a62969071cd944b0825189f4520ad3e1` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:08.189226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141fb0b-6e31-474c-8796-09f1b6b9477f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a62969071cd944b0825189f4520ad3e1)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:08.192749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2141fb0b-6e31-474c-8796-09f1b6b9477f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:08.220414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:08.250298 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2141fb0b-6e31-474c-8796-09f1b6b9477f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a62969071cd944b0825189f4520ad3e1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.251066 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2141fb0b-6e31-474c-8796-09f1b6b9477f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a62969071cd944b0825189f4520ad3e1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.259719 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2141fb0b-6e31-474c-8796-09f1b6b9477f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a62969071cd944b0825189f4520ad3e1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.261726 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2141fb0b-6e31-474c-8796-09f1b6b9477f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a62969071cd944b0825189f4520ad3e1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.279401 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2141fb0b-6e31-474c-8796-09f1b6b9477f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a62969071cd944b0825189f4520ad3e1'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.298261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 935/1863] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:08 2022] DELETE /identity/v3/users/a62969071cd944b0825189f4520ad3e1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:08.632707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:08.634217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:08.651002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389c6b0-0652-4a87-9171-26218e66e60c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:08.709045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945cdf61-023c-4501-a272-21f58c40978e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:08.709619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cdf61-023c-4501-a272-21f58c40978e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:08.709904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cdf61-023c-4501-a272-21f58c40978e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:08.710185 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945cdf61-023c-4501-a272-21f58c40978e admin admin] PATH_INFO: `/v3/projects/a9772b32fd494e89be133523ab9bcbd3` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:08.711372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cdf61-023c-4501-a272-21f58c40978e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9772b32fd494e89be133523ab9bcbd3)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:08.713395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945cdf61-023c-4501-a272-21f58c40978e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:08.771889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:08.783004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-945cdf61-023c-4501-a272-21f58c40978e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9772b32fd494e89be133523ab9bcbd3', 'request_id': 'req-945cdf61-023c-4501-a272-21f58c40978e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.799427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-945cdf61-023c-4501-a272-21f58c40978e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9772b32fd494e89be133523ab9bcbd3', 'request_id': 'req-945cdf61-023c-4501-a272-21f58c40978e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.811175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-945cdf61-023c-4501-a272-21f58c40978e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9772b32fd494e89be133523ab9bcbd3', 'request_id': 'req-945cdf61-023c-4501-a272-21f58c40978e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.837100 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-945cdf61-023c-4501-a272-21f58c40978e admin admin] The token cache is being invalidate because project a9772b32fd494e89be133523ab9bcbd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:08.837608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-945cdf61-023c-4501-a272-21f58c40978e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.838312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-945cdf61-023c-4501-a272-21f58c40978e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:08.840809 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 929/1864] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:08 2022] DELETE /identity/v3/projects/a9772b32fd494e89be133523ab9bcbd3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:08.846884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361502cf-951b-4b1d-8321-fe19aca2a215 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:08.847769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361502cf-951b-4b1d-8321-fe19aca2a215 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:08.848213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361502cf-951b-4b1d-8321-fe19aca2a215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:08.880920 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-361502cf-951b-4b1d-8321-fe19aca2a215 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:08.882168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-361502cf-951b-4b1d-8321-fe19aca2a215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:08.944176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 936/1865] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:08.953442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459c693-c116-4dce-87f2-40c5e1eeeb07 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:08.953731 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459c693-c116-4dce-87f2-40c5e1eeeb07 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:08.954011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459c693-c116-4dce-87f2-40c5e1eeeb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:08.974414 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1459c693-c116-4dce-87f2-40c5e1eeeb07 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:08.975609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1459c693-c116-4dce-87f2-40c5e1eeeb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:08.984067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 930/1866] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:08.988267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361502cf-951b-4b1d-8321-fe19aca2a215 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:08.999450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361502cf-951b-4b1d-8321-fe19aca2a215 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:09.005434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06ad32-556f-4bd3-9b9d-1528bddb3c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:09.006173 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ad32-556f-4bd3-9b9d-1528bddb3c2e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.006521 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ad32-556f-4bd3-9b9d-1528bddb3c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.006905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ad32-556f-4bd3-9b9d-1528bddb3c2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.007864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06ad32-556f-4bd3-9b9d-1528bddb3c2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:09.008244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06ad32-556f-4bd3-9b9d-1528bddb3c2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:09.010002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06ad32-556f-4bd3-9b9d-1528bddb3c2e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:09.023188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 937/1867] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:15:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:15:09.027708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459c693-c116-4dce-87f2-40c5e1eeeb07 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:09.037553 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459c693-c116-4dce-87f2-40c5e1eeeb07 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:09.042911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e788ee4-91c0-48c1-9e5d-e54e4723ff47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:09.043533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e788ee4-91c0-48c1-9e5d-e54e4723ff47 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.043822 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e788ee4-91c0-48c1-9e5d-e54e4723ff47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.044225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e788ee4-91c0-48c1-9e5d-e54e4723ff47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.044906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e788ee4-91c0-48c1-9e5d-e54e4723ff47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:09.046882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e788ee4-91c0-48c1-9e5d-e54e4723ff47 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:09.086570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 931/1868] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:09 2022] POST /identity/v3/projects => generated 382 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:09.089046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06ad32-556f-4bd3-9b9d-1528bddb3c2e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:09.099506 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06ad32-556f-4bd3-9b9d-1528bddb3c2e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:09.161557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f610fa-90c4-41dd-b10b-d538a07d96ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:09.162480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f610fa-90c4-41dd-b10b-d538a07d96ab admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.162580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f610fa-90c4-41dd-b10b-d538a07d96ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.162864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f610fa-90c4-41dd-b10b-d538a07d96ab admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.163548 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f610fa-90c4-41dd-b10b-d538a07d96ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:09.165478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f610fa-90c4-41dd-b10b-d538a07d96ab admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:09.193049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 938/1869] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:09 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:09.194975 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e788ee4-91c0-48c1-9e5d-e54e4723ff47 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:09.206569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e788ee4-91c0-48c1-9e5d-e54e4723ff47 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:09.211517 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec912149-c3d6-43d9-bd52-7f7196ea1227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:09.212140 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec912149-c3d6-43d9-bd52-7f7196ea1227 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.212429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec912149-c3d6-43d9-bd52-7f7196ea1227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.212756 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec912149-c3d6-43d9-bd52-7f7196ea1227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.213461 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec912149-c3d6-43d9-bd52-7f7196ea1227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:09.215121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec912149-c3d6-43d9-bd52-7f7196ea1227 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:09.226275 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 932/1870] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:09 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:09.230958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f610fa-90c4-41dd-b10b-d538a07d96ab admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:09.241179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f610fa-90c4-41dd-b10b-d538a07d96ab admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:09.246755 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ab0ec-3e32-4f2b-82b4-6ec42d634ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:09.247381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ab0ec-3e32-4f2b-82b4-6ec42d634ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.247666 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ab0ec-3e32-4f2b-82b4-6ec42d634ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.247945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ab0ec-3e32-4f2b-82b4-6ec42d634ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.248650 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ab0ec-3e32-4f2b-82b4-6ec42d634ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:09.250365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ab0ec-3e32-4f2b-82b4-6ec42d634ecc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:09.262382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 939/1871] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:09 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:09.267111 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec912149-c3d6-43d9-bd52-7f7196ea1227 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:09.277477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec912149-c3d6-43d9-bd52-7f7196ea1227 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:09.282401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d7479-30cd-493f-b1ac-5ddd5fea7b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:09.283115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d7479-30cd-493f-b1ac-5ddd5fea7b7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.283399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d7479-30cd-493f-b1ac-5ddd5fea7b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.283679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d7479-30cd-493f-b1ac-5ddd5fea7b7e admin admin] PATH_INFO: `/v3/projects/c787724a308d432d832efb60eadfe124/users/f8d01e93971a48bd9f0cc587241746e2/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.299163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d7479-30cd-493f-b1ac-5ddd5fea7b7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c787724a308d432d832efb60eadfe124, user_id=f8d01e93971a48bd9f0cc587241746e2, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:09.301891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d7479-30cd-493f-b1ac-5ddd5fea7b7e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:09.315878 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 933/1872] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:09 2022] PUT /identity/v3/projects/c787724a308d432d832efb60eadfe124/users/f8d01e93971a48bd9f0cc587241746e2/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:09.327515 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f105fa3-ac24-4946-aa47-00856aea2ebd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.327966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f105fa3-ac24-4946-aa47-00856aea2ebd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.328356 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f105fa3-ac24-4946-aa47-00856aea2ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.343557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:09.363355 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2f105fa3-ac24-4946-aa47-00856aea2ebd None None] MFA Rules not processed for user `f8d01e93971a48bd9f0cc587241746e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:09.364989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2f105fa3-ac24-4946-aa47-00856aea2ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:09.436345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 940/1873] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:09.440953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d7479-30cd-493f-b1ac-5ddd5fea7b7e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:09.531464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d7479-30cd-493f-b1ac-5ddd5fea7b7e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:09.536404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13bb57-1b8a-4e0c-b77b-317ccc562190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:09.537043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13bb57-1b8a-4e0c-b77b-317ccc562190 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.537350 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13bb57-1b8a-4e0c-b77b-317ccc562190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.537612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13bb57-1b8a-4e0c-b77b-317ccc562190 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.538311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13bb57-1b8a-4e0c-b77b-317ccc562190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:09.545688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13bb57-1b8a-4e0c-b77b-317ccc562190 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:09.574181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 934/1874] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:09 2022] POST /identity/v3/projects => generated 380 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:09.575464 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f105fa3-ac24-4946-aa47-00856aea2ebd None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:09.587359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f105fa3-ac24-4946-aa47-00856aea2ebd None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:09.648196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130bad5-e6f4-43d4-8ad4-615f8fce7234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:09.648842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130bad5-e6f4-43d4-8ad4-615f8fce7234 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.649136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130bad5-e6f4-43d4-8ad4-615f8fce7234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.649418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130bad5-e6f4-43d4-8ad4-615f8fce7234 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.650674 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130bad5-e6f4-43d4-8ad4-615f8fce7234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:09.652622 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130bad5-e6f4-43d4-8ad4-615f8fce7234 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:09.677239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 941/1875] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:09 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:09.681717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13bb57-1b8a-4e0c-b77b-317ccc562190 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:09.692325 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13bb57-1b8a-4e0c-b77b-317ccc562190 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:09.697037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf11b4f-2ed3-4685-ad91-eb3134ee63ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:09.697676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf11b4f-2ed3-4685-ad91-eb3134ee63ba admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.697962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf11b4f-2ed3-4685-ad91-eb3134ee63ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.698238 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf11b4f-2ed3-4685-ad91-eb3134ee63ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.698978 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf11b4f-2ed3-4685-ad91-eb3134ee63ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:09.700604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf11b4f-2ed3-4685-ad91-eb3134ee63ba admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:09.711822 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 935/1876] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:09 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:09.716807 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130bad5-e6f4-43d4-8ad4-615f8fce7234 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:09.727344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130bad5-e6f4-43d4-8ad4-615f8fce7234 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:09.732566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09c79d-2b02-4c74-bba1-3a263751be33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:09.733238 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09c79d-2b02-4c74-bba1-3a263751be33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.733521 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09c79d-2b02-4c74-bba1-3a263751be33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.733799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09c79d-2b02-4c74-bba1-3a263751be33 admin admin] PATH_INFO: `/v3/projects/54a0743666b3482bb58ac0c4be5ee67e/users/0c112ee257a245cfa5297b424d075cfd/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.749674 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09c79d-2b02-4c74-bba1-3a263751be33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a0743666b3482bb58ac0c4be5ee67e, user_id=0c112ee257a245cfa5297b424d075cfd, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:09.752398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09c79d-2b02-4c74-bba1-3a263751be33 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:09.765992 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 942/1877] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:09 2022] PUT /identity/v3/projects/54a0743666b3482bb58ac0c4be5ee67e/users/0c112ee257a245cfa5297b424d075cfd/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:09.770257 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf11b4f-2ed3-4685-ad91-eb3134ee63ba admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:09.782881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf11b4f-2ed3-4685-ad91-eb3134ee63ba admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:09.815828 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:09.841713 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c9712-4a13-4395-9392-ef6447985bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:09.842340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c9712-4a13-4395-9392-ef6447985bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.842627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c9712-4a13-4395-9392-ef6447985bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.842939 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c9712-4a13-4395-9392-ef6447985bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.843663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c9712-4a13-4395-9392-ef6447985bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:09.845447 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c9712-4a13-4395-9392-ef6447985bc7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:09.856129 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 936/1878] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:09 2022] GET /identity/v3/roles => generated 1427 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:09.860552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09c79d-2b02-4c74-bba1-3a263751be33 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:09.871021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09c79d-2b02-4c74-bba1-3a263751be33 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:09.875843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f702a-3f0b-445c-bc32-8a8210fcdc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:09.876511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f702a-3f0b-445c-bc32-8a8210fcdc61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.876796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f702a-3f0b-445c-bc32-8a8210fcdc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.877072 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f702a-3f0b-445c-bc32-8a8210fcdc61 admin admin] PATH_INFO: `/v3/projects/54a0743666b3482bb58ac0c4be5ee67e/users/0c112ee257a245cfa5297b424d075cfd/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.879056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f702a-3f0b-445c-bc32-8a8210fcdc61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a0743666b3482bb58ac0c4be5ee67e, user_id=0c112ee257a245cfa5297b424d075cfd, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:09.881687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f702a-3f0b-445c-bc32-8a8210fcdc61 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:09.892810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 943/1879] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:09 2022] PUT /identity/v3/projects/54a0743666b3482bb58ac0c4be5ee67e/users/0c112ee257a245cfa5297b424d075cfd/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:09.904600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067680eb-a7e7-43bb-9442-df2b4bad3bbc None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:09.904891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067680eb-a7e7-43bb-9442-df2b4bad3bbc None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:09.905180 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067680eb-a7e7-43bb-9442-df2b4bad3bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:09.935634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-067680eb-a7e7-43bb-9442-df2b4bad3bbc None None] MFA Rules not processed for user `0c112ee257a245cfa5297b424d075cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:09.936881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-067680eb-a7e7-43bb-9442-df2b4bad3bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:10.013696 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 937/1880] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:10.024537 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f702a-3f0b-445c-bc32-8a8210fcdc61 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:10.027051 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-df8f702a-3f0b-445c-bc32-8a8210fcdc61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:10.084945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:10.090204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f702a-3f0b-445c-bc32-8a8210fcdc61 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:10.095073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07e54e-1234-41a3-a574-7af1bc169686 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:10.095706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07e54e-1234-41a3-a574-7af1bc169686 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:10.095990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07e54e-1234-41a3-a574-7af1bc169686 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:10.096270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07e54e-1234-41a3-a574-7af1bc169686 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:10.108077 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07e54e-1234-41a3-a574-7af1bc169686 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:10.109582 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07e54e-1234-41a3-a574-7af1bc169686 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:10.120484 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 944/1881] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:10.583830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067680eb-a7e7-43bb-9442-df2b4bad3bbc None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:10.658258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067680eb-a7e7-43bb-9442-df2b4bad3bbc None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:10.663135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243627b-7ed6-4a73-b66b-1fc176b94907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:10.663795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243627b-7ed6-4a73-b66b-1fc176b94907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:10.664083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243627b-7ed6-4a73-b66b-1fc176b94907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:10.664383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243627b-7ed6-4a73-b66b-1fc176b94907 admin admin] PATH_INFO: `/v3/users/f8d01e93971a48bd9f0cc587241746e2` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:10.666308 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243627b-7ed6-4a73-b66b-1fc176b94907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8d01e93971a48bd9f0cc587241746e2)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:10.668500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243627b-7ed6-4a73-b66b-1fc176b94907 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:10.726532 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0243627b-7ed6-4a73-b66b-1fc176b94907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8d01e93971a48bd9f0cc587241746e2'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:10.737182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0243627b-7ed6-4a73-b66b-1fc176b94907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8d01e93971a48bd9f0cc587241746e2'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:10.737919 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0243627b-7ed6-4a73-b66b-1fc176b94907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8d01e93971a48bd9f0cc587241746e2'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:10.738542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0243627b-7ed6-4a73-b66b-1fc176b94907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8d01e93971a48bd9f0cc587241746e2'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:10.745673 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:10.754967 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0243627b-7ed6-4a73-b66b-1fc176b94907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8d01e93971a48bd9f0cc587241746e2'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:10.769052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 938/1882] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:10 2022] DELETE /identity/v3/users/f8d01e93971a48bd9f0cc587241746e2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:10.773430 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf1d34-2761-4c5f-8a77-755c8c9615c0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:10.773934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf1d34-2761-4c5f-8a77-755c8c9615c0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:10.774034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf1d34-2761-4c5f-8a77-755c8c9615c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:10.794003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-debf1d34-2761-4c5f-8a77-755c8c9615c0 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:10.795295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-debf1d34-2761-4c5f-8a77-755c8c9615c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:10.858369 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 945/1883] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:10.868101 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243627b-7ed6-4a73-b66b-1fc176b94907 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:10.869616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0243627b-7ed6-4a73-b66b-1fc176b94907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:10.926810 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243627b-7ed6-4a73-b66b-1fc176b94907 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:10.930412 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024141a-2e5b-4e1c-b315-b051f91e21ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:10.931065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024141a-2e5b-4e1c-b315-b051f91e21ec service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:10.931377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024141a-2e5b-4e1c-b315-b051f91e21ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:10.931655 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024141a-2e5b-4e1c-b315-b051f91e21ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:10.940793 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024141a-2e5b-4e1c-b315-b051f91e21ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:10.942247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024141a-2e5b-4e1c-b315-b051f91e21ec service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:10.952083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 939/1884] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:11.220582 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf1d34-2761-4c5f-8a77-755c8c9615c0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:11.222087 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-debf1d34-2761-4c5f-8a77-755c8c9615c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:11.241796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf1d34-2761-4c5f-8a77-755c8c9615c0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:11.248299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:11.250741 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:11.251132 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:11.251512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] PATH_INFO: `/v3/projects/c787724a308d432d832efb60eadfe124` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:11.252994 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c787724a308d432d832efb60eadfe124)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:11.255435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:11.309864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c787724a308d432d832efb60eadfe124', 'request_id': 'req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:11.315881 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c787724a308d432d832efb60eadfe124', 'request_id': 'req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:11.321307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c787724a308d432d832efb60eadfe124', 'request_id': 'req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:11.334963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] The token cache is being invalidate because project c787724a308d432d832efb60eadfe124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:11.335255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:11.335887 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:11.337456 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 946/1885] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:11 2022] DELETE /identity/v3/projects/c787724a308d432d832efb60eadfe124 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:11.342156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024141a-2e5b-4e1c-b315-b051f91e21ec service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:11.352566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024141a-2e5b-4e1c-b315-b051f91e21ec service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:11.410143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ac35b-807e-4aad-9ae9-33878b11e521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:11.410817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ac35b-807e-4aad-9ae9-33878b11e521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:11.411104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ac35b-807e-4aad-9ae9-33878b11e521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:11.411383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ac35b-807e-4aad-9ae9-33878b11e521 admin admin] PATH_INFO: `/v3/users/0c112ee257a245cfa5297b424d075cfd` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:11.412548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ac35b-807e-4aad-9ae9-33878b11e521 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c112ee257a245cfa5297b424d075cfd)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:11.414474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ac35b-807e-4aad-9ae9-33878b11e521 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:11.459571 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9f7ac35b-807e-4aad-9ae9-33878b11e521 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c112ee257a245cfa5297b424d075cfd'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:11.467717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9f7ac35b-807e-4aad-9ae9-33878b11e521 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c112ee257a245cfa5297b424d075cfd'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:11.468361 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9f7ac35b-807e-4aad-9ae9-33878b11e521 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c112ee257a245cfa5297b424d075cfd'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:11.468987 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9f7ac35b-807e-4aad-9ae9-33878b11e521 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c112ee257a245cfa5297b424d075cfd'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:11.480942 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9f7ac35b-807e-4aad-9ae9-33878b11e521 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c112ee257a245cfa5297b424d075cfd'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:11.491001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:11.493809 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 940/1886] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:11 2022] DELETE /identity/v3/users/0c112ee257a245cfa5297b424d075cfd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:11.793333 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:11.795450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:11.810980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79940ca9-892a-46a2-9884-26bf472a2257 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:11.811913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79940ca9-892a-46a2-9884-26bf472a2257 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:11.812332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79940ca9-892a-46a2-9884-26bf472a2257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:11.829816 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7c4e8-3af0-4c02-9a9a-861519768ff4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:11.852738 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-79940ca9-892a-46a2-9884-26bf472a2257 None None] MFA Rules not processed for user `e7f757eabcb84de6ab042e0ac5a8158c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:11.854022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-79940ca9-892a-46a2-9884-26bf472a2257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:11.910581 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:11.911294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:11.911578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:11.911859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] PATH_INFO: `/v3/projects/54a0743666b3482bb58ac0c4be5ee67e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:11.913095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=54a0743666b3482bb58ac0c4be5ee67e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:11.915236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:11.932829 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 941/1887] 10.222.0.97 () {58 vars in 1019 bytes} [Mon Aug 1 15:15:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:11.939930 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:11.962044 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79940ca9-892a-46a2-9884-26bf472a2257 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:11.964197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-79940ca9-892a-46a2-9884-26bf472a2257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:11.981045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79940ca9-892a-46a2-9884-26bf472a2257 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:11.998322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54a0743666b3482bb58ac0c4be5ee67e', 'request_id': 'req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:12.005159 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54a0743666b3482bb58ac0c4be5ee67e', 'request_id': 'req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:12.013187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54a0743666b3482bb58ac0c4be5ee67e', 'request_id': 'req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:12.036883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] The token cache is being invalidate because project 54a0743666b3482bb58ac0c4be5ee67e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:12.037052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:12.037777 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:12.039446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 947/1888] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:11 2022] DELETE /identity/v3/projects/54a0743666b3482bb58ac0c4be5ee67e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:12.044138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:12.045941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:12.067848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfbc88-fffe-4dbe-b8d9-af2cadbbaee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:12.068938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfbc88-fffe-4dbe-b8d9-af2cadbbaee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:12.069223 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfbc88-fffe-4dbe-b8d9-af2cadbbaee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:12.069685 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfbc88-fffe-4dbe-b8d9-af2cadbbaee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:12.071465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3abfbc88-fffe-4dbe-b8d9-af2cadbbaee5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:12.081775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfbc88-fffe-4dbe-b8d9-af2cadbbaee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:12.084147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfbc88-fffe-4dbe-b8d9-af2cadbbaee5 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:12.093150 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 942/1889] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:12.093984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfbc88-fffe-4dbe-b8d9-af2cadbbaee5 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:12.098159 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3abfbc88-fffe-4dbe-b8d9-af2cadbbaee5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:12.109288 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfbc88-fffe-4dbe-b8d9-af2cadbbaee5 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:12.113534 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87b115-0477-4e37-bc89-b658d5154dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:12.114157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87b115-0477-4e37-bc89-b658d5154dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:12.114438 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87b115-0477-4e37-bc89-b658d5154dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:12.114774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87b115-0477-4e37-bc89-b658d5154dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:12.116165 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f908de7-df93-401d-b7f8-b1fd07e5c9ff admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:12.123028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1b66b-5432-41e3-b0d3-23dbe77f90ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:12.123874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87b115-0477-4e37-bc89-b658d5154dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:12.123996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1b66b-5432-41e3-b0d3-23dbe77f90ee service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:12.124374 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1b66b-5432-41e3-b0d3-23dbe77f90ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:12.124753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1b66b-5432-41e3-b0d3-23dbe77f90ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:12.125627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87b115-0477-4e37-bc89-b658d5154dc4 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:12.135425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1b66b-5432-41e3-b0d3-23dbe77f90ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:12.138155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1b66b-5432-41e3-b0d3-23dbe77f90ee service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:12.145961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 943/1890] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:12.149323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 948/1891] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:12.150024 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bb885-95e7-4a02-ad8f-81f9082f7607 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:12.150437 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bb885-95e7-4a02-ad8f-81f9082f7607 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:12.150705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0bb885-95e7-4a02-ad8f-81f9082f7607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:12.174485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-de0bb885-95e7-4a02-ad8f-81f9082f7607 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:12.175878 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-de0bb885-95e7-4a02-ad8f-81f9082f7607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:12.245816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 944/1892] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:12.252796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a0839-b442-4be5-a292-234e2bfd1904 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:12.256440 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a0839-b442-4be5-a292-234e2bfd1904 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:12.256862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a0839-b442-4be5-a292-234e2bfd1904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:12.281031 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-482a0839-b442-4be5-a292-234e2bfd1904 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:12.282714 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-482a0839-b442-4be5-a292-234e2bfd1904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:12.320832 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 949/1893] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:12.321779 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0bb885-95e7-4a02-ad8f-81f9082f7607 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:12.347123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0bb885-95e7-4a02-ad8f-81f9082f7607 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:12.355462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692f50f-f750-4bf3-8652-a11d84549f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:12.356112 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692f50f-f750-4bf3-8652-a11d84549f02 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:12.356401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692f50f-f750-4bf3-8652-a11d84549f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:12.356679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692f50f-f750-4bf3-8652-a11d84549f02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:12.357393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692f50f-f750-4bf3-8652-a11d84549f02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:12.357896 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692f50f-f750-4bf3-8652-a11d84549f02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:12.359431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a692f50f-f750-4bf3-8652-a11d84549f02 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:12.395970 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 945/1894] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:15:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:15:12.398422 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a0839-b442-4be5-a292-234e2bfd1904 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:12.418088 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a0839-b442-4be5-a292-234e2bfd1904 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:12.433615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70d3b7-eaeb-4147-98ea-047796f3ad69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:12.434710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70d3b7-eaeb-4147-98ea-047796f3ad69 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:12.435220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70d3b7-eaeb-4147-98ea-047796f3ad69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:12.435451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70d3b7-eaeb-4147-98ea-047796f3ad69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:12.436134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c70d3b7-eaeb-4147-98ea-047796f3ad69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:12.438329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c70d3b7-eaeb-4147-98ea-047796f3ad69 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:12.483983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 950/1895] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:12 2022] POST /identity/v3/projects => generated 374 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:12.487445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692f50f-f750-4bf3-8652-a11d84549f02 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:12.502903 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a692f50f-f750-4bf3-8652-a11d84549f02 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:12.533395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:12.651143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70084d55-b181-4837-af50-fd73c1aa5d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:12.653297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70084d55-b181-4837-af50-fd73c1aa5d1f admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:12.653608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70084d55-b181-4837-af50-fd73c1aa5d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:12.655083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70084d55-b181-4837-af50-fd73c1aa5d1f admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:12.656650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70084d55-b181-4837-af50-fd73c1aa5d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:12.660881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70084d55-b181-4837-af50-fd73c1aa5d1f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:12.706831 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 946/1896] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:12 2022] POST /identity/v3/users => generated 288 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:12.717758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70d3b7-eaeb-4147-98ea-047796f3ad69 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:12.734174 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70d3b7-eaeb-4147-98ea-047796f3ad69 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:12.741558 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6efb4-9983-47f4-9042-cdfe0a3469a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:12.742556 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6efb4-9983-47f4-9042-cdfe0a3469a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:12.743391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6efb4-9983-47f4-9042-cdfe0a3469a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:12.744176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6efb4-9983-47f4-9042-cdfe0a3469a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:12.745256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6efb4-9983-47f4-9042-cdfe0a3469a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:12.749272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6efb4-9983-47f4-9042-cdfe0a3469a4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:12.776227 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 951/1897] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:12 2022] GET /identity/v3/roles => generated 1427 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:12.777875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70084d55-b181-4837-af50-fd73c1aa5d1f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:12.792422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70084d55-b181-4837-af50-fd73c1aa5d1f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:12.803095 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687956d-3d53-43bd-a0ca-0ba5f26283a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:12.804181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687956d-3d53-43bd-a0ca-0ba5f26283a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:12.805232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687956d-3d53-43bd-a0ca-0ba5f26283a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:12.805700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687956d-3d53-43bd-a0ca-0ba5f26283a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:12.806718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687956d-3d53-43bd-a0ca-0ba5f26283a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:12.809449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687956d-3d53-43bd-a0ca-0ba5f26283a1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:12.842945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6efb4-9983-47f4-9042-cdfe0a3469a4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:12.852448 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 947/1898] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:12 2022] GET /identity/v3/roles => generated 1427 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:12.859740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6efb4-9983-47f4-9042-cdfe0a3469a4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:12.868875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b681a36-e8a0-41f9-bc62-e98ee4768913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:12.869752 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b681a36-e8a0-41f9-bc62-e98ee4768913 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:12.870182 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b681a36-e8a0-41f9-bc62-e98ee4768913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:12.870829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b681a36-e8a0-41f9-bc62-e98ee4768913 admin admin] PATH_INFO: `/v3/projects/8496ce7ee93548e59749725a0ff83c6d/users/143357a3682a4f118c7868e2feca3037/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:12.891020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b681a36-e8a0-41f9-bc62-e98ee4768913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8496ce7ee93548e59749725a0ff83c6d, user_id=143357a3682a4f118c7868e2feca3037, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:12.894513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b681a36-e8a0-41f9-bc62-e98ee4768913 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:12.938949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 952/1899] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:12 2022] PUT /identity/v3/projects/8496ce7ee93548e59749725a0ff83c6d/users/143357a3682a4f118c7868e2feca3037/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:12.957890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce5690-eb8d-411f-a730-1dc03432f2af None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:12.958190 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce5690-eb8d-411f-a730-1dc03432f2af None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:12.958456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce5690-eb8d-411f-a730-1dc03432f2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:12.997868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-27ce5690-eb8d-411f-a730-1dc03432f2af None None] MFA Rules not processed for user `143357a3682a4f118c7868e2feca3037`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:12.999274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-27ce5690-eb8d-411f-a730-1dc03432f2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:13.107261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 948/1900] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:13.118217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b681a36-e8a0-41f9-bc62-e98ee4768913 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:13.120799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6b681a36-e8a0-41f9-bc62-e98ee4768913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:13.251215 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:13.274794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b681a36-e8a0-41f9-bc62-e98ee4768913 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:13.283315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643f933-1c1a-40bf-81c2-804c10ceb4ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:13.283967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643f933-1c1a-40bf-81c2-804c10ceb4ac service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:13.284265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643f933-1c1a-40bf-81c2-804c10ceb4ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:13.284645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643f933-1c1a-40bf-81c2-804c10ceb4ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:13.310417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2643f933-1c1a-40bf-81c2-804c10ceb4ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:13.317087 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2643f933-1c1a-40bf-81c2-804c10ceb4ac service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:13.329753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 953/1901] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:13.357039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce5690-eb8d-411f-a730-1dc03432f2af None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:13.496492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce5690-eb8d-411f-a730-1dc03432f2af None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:13.508986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be703fbc-7d27-41e7-9711-ff3ab8e9bf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:13.509817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be703fbc-7d27-41e7-9711-ff3ab8e9bf1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:13.510175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be703fbc-7d27-41e7-9711-ff3ab8e9bf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:13.510523 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be703fbc-7d27-41e7-9711-ff3ab8e9bf1b admin admin] PATH_INFO: `/v3/users/143357a3682a4f118c7868e2feca3037` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:13.512057 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be703fbc-7d27-41e7-9711-ff3ab8e9bf1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=143357a3682a4f118c7868e2feca3037)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:13.514557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be703fbc-7d27-41e7-9711-ff3ab8e9bf1b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:13.546442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd706a-023c-4a0f-9fa3-02991a5e4549 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:13.546442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd706a-023c-4a0f-9fa3-02991a5e4549 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:13.546442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd706a-023c-4a0f-9fa3-02991a5e4549 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:13.546863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 954/1902] 10.222.0.100 () {54 vars in 926 bytes} [Mon Aug 1 15:15:13 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:15:13.563931 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c9e11-3962-4b8d-a2f5-d96e74979f89 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:13.566318 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c9e11-3962-4b8d-a2f5-d96e74979f89 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:13.566701 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c9e11-3962-4b8d-a2f5-d96e74979f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:13.621687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e60c9e11-3962-4b8d-a2f5-d96e74979f89 None None] MFA Rules not processed for user `ec50ac7c54324884a1b2a6c63d617907`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:13.624693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e60c9e11-3962-4b8d-a2f5-d96e74979f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:13.637384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:13.679491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-be703fbc-7d27-41e7-9711-ff3ab8e9bf1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '143357a3682a4f118c7868e2feca3037'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:13.701492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-be703fbc-7d27-41e7-9711-ff3ab8e9bf1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '143357a3682a4f118c7868e2feca3037'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:13.702512 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-be703fbc-7d27-41e7-9711-ff3ab8e9bf1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '143357a3682a4f118c7868e2feca3037'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:13.704192 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-be703fbc-7d27-41e7-9711-ff3ab8e9bf1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '143357a3682a4f118c7868e2feca3037'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:13.751680 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-be703fbc-7d27-41e7-9711-ff3ab8e9bf1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '143357a3682a4f118c7868e2feca3037'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:13.796351 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 949/1903] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:13 2022] DELETE /identity/v3/users/143357a3682a4f118c7868e2feca3037 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:13.800694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ec7bc-d0fb-46a9-a084-930bf8ce3f98 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:13.801553 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ec7bc-d0fb-46a9-a084-930bf8ce3f98 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:13.803420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ec7bc-d0fb-46a9-a084-930bf8ce3f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:13.846056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 955/1904] 10.222.0.100 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:13.849857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-173ec7bc-d0fb-46a9-a084-930bf8ce3f98 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:13.851420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-173ec7bc-d0fb-46a9-a084-930bf8ce3f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:13.894369 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c9e11-3962-4b8d-a2f5-d96e74979f89 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:13.901530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e60c9e11-3962-4b8d-a2f5-d96e74979f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:13.933249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c9e11-3962-4b8d-a2f5-d96e74979f89 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:14.104372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 950/1905] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:14.121797 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e6ea4-e1cb-420b-b71a-2b16cb0f013f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:14.123004 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e6ea4-e1cb-420b-b71a-2b16cb0f013f service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:14.123474 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e6ea4-e1cb-420b-b71a-2b16cb0f013f service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:14.123992 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e6ea4-e1cb-420b-b71a-2b16cb0f013f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:14.138460 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e6ea4-e1cb-420b-b71a-2b16cb0f013f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:14.140229 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e6ea4-e1cb-420b-b71a-2b16cb0f013f service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:14.143607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ec7bc-d0fb-46a9-a084-930bf8ce3f98 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:14.145156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-173ec7bc-d0fb-46a9-a084-930bf8ce3f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:14.161524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ec7bc-d0fb-46a9-a084-930bf8ce3f98 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:14.168128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0df628-da5d-47fb-b6d0-a36454afb0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:14.169045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0df628-da5d-47fb-b6d0-a36454afb0c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:14.169045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0df628-da5d-47fb-b6d0-a36454afb0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:14.169466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0df628-da5d-47fb-b6d0-a36454afb0c8 admin admin] PATH_INFO: `/v3/users/7f02f57d4ea04ec4a92f76e35afe6592` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:14.171266 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 956/1906] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:14.171459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0df628-da5d-47fb-b6d0-a36454afb0c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f02f57d4ea04ec4a92f76e35afe6592)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:14.173674 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0df628-da5d-47fb-b6d0-a36454afb0c8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:14.203707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e6ea4-e1cb-420b-b71a-2b16cb0f013f service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:14.209272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-307e6ea4-e1cb-420b-b71a-2b16cb0f013f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:14.261706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1a0df628-da5d-47fb-b6d0-a36454afb0c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f02f57d4ea04ec4a92f76e35afe6592'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:14.276883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1a0df628-da5d-47fb-b6d0-a36454afb0c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f02f57d4ea04ec4a92f76e35afe6592'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:14.277535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1a0df628-da5d-47fb-b6d0-a36454afb0c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f02f57d4ea04ec4a92f76e35afe6592'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:14.278202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1a0df628-da5d-47fb-b6d0-a36454afb0c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f02f57d4ea04ec4a92f76e35afe6592'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:14.307846 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1a0df628-da5d-47fb-b6d0-a36454afb0c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f02f57d4ea04ec4a92f76e35afe6592'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:14.336741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 951/1907] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:14 2022] DELETE /identity/v3/users/7f02f57d4ea04ec4a92f76e35afe6592 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:14.343108 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2bd05-90e8-4d13-9787-8c27d4690093 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:14.343460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2bd05-90e8-4d13-9787-8c27d4690093 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:14.343685 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f2bd05-90e8-4d13-9787-8c27d4690093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:14.356610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e6ea4-e1cb-420b-b71a-2b16cb0f013f service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:14.387708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-71f2bd05-90e8-4d13-9787-8c27d4690093 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:14.389184 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-71f2bd05-90e8-4d13-9787-8c27d4690093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:14.429821 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:14.450629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:14.520654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4a0e0-0442-478d-a20b-8578b96c2e69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:14.521290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4a0e0-0442-478d-a20b-8578b96c2e69 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:14.521570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4a0e0-0442-478d-a20b-8578b96c2e69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:14.522434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4a0e0-0442-478d-a20b-8578b96c2e69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:14.524653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-13c4a0e0-0442-478d-a20b-8578b96c2e69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:14.580521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 952/1908] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:14.609984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2bd05-90e8-4d13-9787-8c27d4690093 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:14.612093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-71f2bd05-90e8-4d13-9787-8c27d4690093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:14.638179 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f2bd05-90e8-4d13-9787-8c27d4690093 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:14.643751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126862d-1b56-43d2-ba26-8b0b25bcd89c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:14.645427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126862d-1b56-43d2-ba26-8b0b25bcd89c service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:14.646995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126862d-1b56-43d2-ba26-8b0b25bcd89c service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:14.647695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126862d-1b56-43d2-ba26-8b0b25bcd89c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:14.663366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126862d-1b56-43d2-ba26-8b0b25bcd89c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:14.665128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4126862d-1b56-43d2-ba26-8b0b25bcd89c service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:14.694150 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 953/1909] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:14.700067 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4a0e0-0442-478d-a20b-8578b96c2e69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:14.701813 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c4a0e0-0442-478d-a20b-8578b96c2e69 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:14.714902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 957/1910] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:14.936671 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0f485-df75-4b00-ad8d-960faf740dfe None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:14.937445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0f485-df75-4b00-ad8d-960faf740dfe None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:14.943203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0f485-df75-4b00-ad8d-960faf740dfe None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:14.945107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 954/1911] 10.222.0.100 () {54 vars in 926 bytes} [Mon Aug 1 15:15:14 2022] GET /identity => generated 271 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:15:14.975226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b33174-a8b9-413e-ba29-ee1da9bbb2b3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:14.975755 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b33174-a8b9-413e-ba29-ee1da9bbb2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:14.975755 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b33174-a8b9-413e-ba29-ee1da9bbb2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:15.030239 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-98b33174-a8b9-413e-ba29-ee1da9bbb2b3 None None] MFA Rules not processed for user `ec50ac7c54324884a1b2a6c63d617907`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:15.040818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-98b33174-a8b9-413e-ba29-ee1da9bbb2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:15.192793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 958/1912] 10.222.0.100 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:15.251826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0f485-df75-4b00-ad8d-960faf740dfe None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:15.255319 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-51a0f485-df75-4b00-ad8d-960faf740dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:15.282511 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0f485-df75-4b00-ad8d-960faf740dfe None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:15.289928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ffca4-6341-4672-a4be-d3581fe981c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:15.290647 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ffca4-6341-4672-a4be-d3581fe981c0 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:15.291064 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ffca4-6341-4672-a4be-d3581fe981c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:15.291453 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ffca4-6341-4672-a4be-d3581fe981c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:15.304668 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ffca4-6341-4672-a4be-d3581fe981c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:15.306384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ffca4-6341-4672-a4be-d3581fe981c0 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:15.320781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 955/1913] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:15.720587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b33174-a8b9-413e-ba29-ee1da9bbb2b3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:15.722638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-98b33174-a8b9-413e-ba29-ee1da9bbb2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:15.749880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b33174-a8b9-413e-ba29-ee1da9bbb2b3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:15.766819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:15.767268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:15.767688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:15.768097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] PATH_INFO: `/v3/projects/922ec89747ef43d0ab18b0e7ef7b5b39` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:15.779943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] RBAC: Authorizing `identity:delete_project(project_id=922ec89747ef43d0ab18b0e7ef7b5b39)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:15.782860 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:15.899481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '922ec89747ef43d0ab18b0e7ef7b5b39', 'request_id': 'req-1a438425-8a44-4cec-b4b1-ba261435f98d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:15.920690 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ffca4-6341-4672-a4be-d3581fe981c0 service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:15.922964 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b89ffca4-6341-4672-a4be-d3581fe981c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:15.943062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '922ec89747ef43d0ab18b0e7ef7b5b39', 'request_id': 'req-1a438425-8a44-4cec-b4b1-ba261435f98d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:15.948722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:15.960141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ffca4-6341-4672-a4be-d3581fe981c0 service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:15.961148 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '922ec89747ef43d0ab18b0e7ef7b5b39', 'request_id': 'req-1a438425-8a44-4cec-b4b1-ba261435f98d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:15.995075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] The token cache is being invalidate because project 922ec89747ef43d0ab18b0e7ef7b5b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:15.996300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:15.997486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.003582 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 959/1914] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:15 2022] DELETE /identity/v3/projects/922ec89747ef43d0ab18b0e7ef7b5b39 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:16.004518 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:16.006335 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:16.083823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:16.085666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.086182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.086650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] PATH_INFO: `/v3/projects/8496ce7ee93548e59749725a0ff83c6d` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.092789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8496ce7ee93548e59749725a0ff83c6d)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:16.096170 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:16.124261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a438425-8a44-4cec-b4b1-ba261435f98d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:16.129517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37f337-cab5-4ebe-aa9a-f4031cf1af09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:16.130372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37f337-cab5-4ebe-aa9a-f4031cf1af09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.130782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37f337-cab5-4ebe-aa9a-f4031cf1af09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.131168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37f337-cab5-4ebe-aa9a-f4031cf1af09 admin admin] PATH_INFO: `/v3/users/20df6d114ac24ea497df923011007f1b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.132403 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d37f337-cab5-4ebe-aa9a-f4031cf1af09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20df6d114ac24ea497df923011007f1b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:16.135092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d37f337-cab5-4ebe-aa9a-f4031cf1af09 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:16.157549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8496ce7ee93548e59749725a0ff83c6d', 'request_id': 'req-311b80b1-d42f-4ca7-ae50-925f17a4116e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.178595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8496ce7ee93548e59749725a0ff83c6d', 'request_id': 'req-311b80b1-d42f-4ca7-ae50-925f17a4116e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.191507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8496ce7ee93548e59749725a0ff83c6d', 'request_id': 'req-311b80b1-d42f-4ca7-ae50-925f17a4116e'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.199573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d37f337-cab5-4ebe-aa9a-f4031cf1af09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20df6d114ac24ea497df923011007f1b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.203207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d37f337-cab5-4ebe-aa9a-f4031cf1af09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20df6d114ac24ea497df923011007f1b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.217131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d37f337-cab5-4ebe-aa9a-f4031cf1af09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20df6d114ac24ea497df923011007f1b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.218076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d37f337-cab5-4ebe-aa9a-f4031cf1af09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20df6d114ac24ea497df923011007f1b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.218999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] The token cache is being invalidate because project 8496ce7ee93548e59749725a0ff83c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:16.219841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.220879 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.222787 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 956/1915] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:15 2022] DELETE /identity/v3/projects/8496ce7ee93548e59749725a0ff83c6d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:16.223838 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:16.225587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:16.234530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5d37f337-cab5-4ebe-aa9a-f4031cf1af09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20df6d114ac24ea497df923011007f1b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.257261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 960/1916] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:16 2022] DELETE /identity/v3/users/20df6d114ac24ea497df923011007f1b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:16.260163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ef9c7-5715-43ca-89cd-ea6a602bce98 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.260615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ef9c7-5715-43ca-89cd-ea6a602bce98 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.261104 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ef9c7-5715-43ca-89cd-ea6a602bce98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.275152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:16.306867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-da5ef9c7-5715-43ca-89cd-ea6a602bce98 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:16.308114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-da5ef9c7-5715-43ca-89cd-ea6a602bce98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:16.322772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b80b1-d42f-4ca7-ae50-925f17a4116e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:16.413154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68770d28-b681-4412-a7b7-daa3d7cee7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:16.413863 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68770d28-b681-4412-a7b7-daa3d7cee7eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.414137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68770d28-b681-4412-a7b7-daa3d7cee7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.414422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68770d28-b681-4412-a7b7-daa3d7cee7eb admin admin] PATH_INFO: `/v3/users/18b5c2efef694a5cbbd2ec60f7ad30aa` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.416174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68770d28-b681-4412-a7b7-daa3d7cee7eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=18b5c2efef694a5cbbd2ec60f7ad30aa)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:16.417770 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 961/1917] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:16.418160 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68770d28-b681-4412-a7b7-daa3d7cee7eb admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:16.426173 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ee1d8-ac37-4c3c-83fd-d004b28c64e9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.426386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ee1d8-ac37-4c3c-83fd-d004b28c64e9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.426675 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ee1d8-ac37-4c3c-83fd-d004b28c64e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.438034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:16.451848 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-271ee1d8-ac37-4c3c-83fd-d004b28c64e9 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:16.453352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-271ee1d8-ac37-4c3c-83fd-d004b28c64e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:16.465091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 962/1918] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:16.469972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ee1d8-ac37-4c3c-83fd-d004b28c64e9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:16.480487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ee1d8-ac37-4c3c-83fd-d004b28c64e9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:16.480696 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-68770d28-b681-4412-a7b7-daa3d7cee7eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18b5c2efef694a5cbbd2ec60f7ad30aa'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.486226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61373756-a43f-4443-a7e7-7347f3802f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:16.486904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61373756-a43f-4443-a7e7-7347f3802f34 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.487166 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61373756-a43f-4443-a7e7-7347f3802f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.487445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61373756-a43f-4443-a7e7-7347f3802f34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.490844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61373756-a43f-4443-a7e7-7347f3802f34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:16.490844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61373756-a43f-4443-a7e7-7347f3802f34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:16.490844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61373756-a43f-4443-a7e7-7347f3802f34 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:16.493020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-68770d28-b681-4412-a7b7-daa3d7cee7eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18b5c2efef694a5cbbd2ec60f7ad30aa'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.494547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-68770d28-b681-4412-a7b7-daa3d7cee7eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18b5c2efef694a5cbbd2ec60f7ad30aa'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.495223 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-68770d28-b681-4412-a7b7-daa3d7cee7eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18b5c2efef694a5cbbd2ec60f7ad30aa'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.509912 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-68770d28-b681-4412-a7b7-daa3d7cee7eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18b5c2efef694a5cbbd2ec60f7ad30aa'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.514235 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 963/1919] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:15:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:15:16.518743 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61373756-a43f-4443-a7e7-7347f3802f34 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:16.521327 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-61373756-a43f-4443-a7e7-7347f3802f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:16.524993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 957/1920] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:16 2022] DELETE /identity/v3/users/18b5c2efef694a5cbbd2ec60f7ad30aa => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:16.529684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f55d9b-f0e9-488e-95e0-d562ba3c7499 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.530215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f55d9b-f0e9-488e-95e0-d562ba3c7499 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.530728 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f55d9b-f0e9-488e-95e0-d562ba3c7499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.535892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61373756-a43f-4443-a7e7-7347f3802f34 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:16.564603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-26f55d9b-f0e9-488e-95e0-d562ba3c7499 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:16.566364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-26f55d9b-f0e9-488e-95e0-d562ba3c7499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:16.604684 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d24c44-808d-4239-a014-44832b1d70fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:16.605377 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d24c44-808d-4239-a014-44832b1d70fc admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.605671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d24c44-808d-4239-a014-44832b1d70fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.606078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d24c44-808d-4239-a014-44832b1d70fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.606804 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d24c44-808d-4239-a014-44832b1d70fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:16.608814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d24c44-808d-4239-a014-44832b1d70fc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:16.639444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:16.654697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 958/1921] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:16.656021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 964/1922] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:16 2022] POST /identity/v3/projects => generated 390 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:16.658998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f55d9b-f0e9-488e-95e0-d562ba3c7499 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:16.659758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d24c44-808d-4239-a014-44832b1d70fc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:16.661753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-26f55d9b-f0e9-488e-95e0-d562ba3c7499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:16.682260 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f55d9b-f0e9-488e-95e0-d562ba3c7499 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:16.735607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d24c44-808d-4239-a014-44832b1d70fc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:16.742991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17710b-71c0-4323-93ab-89ffa1c054a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:16.744248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17710b-71c0-4323-93ab-89ffa1c054a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.744529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17710b-71c0-4323-93ab-89ffa1c054a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.744972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17710b-71c0-4323-93ab-89ffa1c054a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.745882 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17710b-71c0-4323-93ab-89ffa1c054a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:16.746225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:16.746924 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.747202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.747482 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] PATH_INFO: `/v3/projects/bccaadc383f845bfa8247425217facfa` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.748679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bccaadc383f845bfa8247425217facfa)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:16.748788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17710b-71c0-4323-93ab-89ffa1c054a9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:16.750622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:16.778752 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 965/1923] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:16 2022] POST /identity/v3/users => generated 296 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:16.780034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17710b-71c0-4323-93ab-89ffa1c054a9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:16.782234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7e17710b-71c0-4323-93ab-89ffa1c054a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:16.804875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bccaadc383f845bfa8247425217facfa', 'request_id': 'req-543a5f6b-8579-4522-b33d-8eba0059e076'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.813281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bccaadc383f845bfa8247425217facfa', 'request_id': 'req-543a5f6b-8579-4522-b33d-8eba0059e076'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.823400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bccaadc383f845bfa8247425217facfa', 'request_id': 'req-543a5f6b-8579-4522-b33d-8eba0059e076'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.836582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] The token cache is being invalidate because project bccaadc383f845bfa8247425217facfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:16.836969 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.838257 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:16.843149 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 959/1924] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:16 2022] DELETE /identity/v3/projects/bccaadc383f845bfa8247425217facfa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:16.843149 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:16.845070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:16.857306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17710b-71c0-4323-93ab-89ffa1c054a9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:16.869543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:16.913260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54600da1-fdfa-44b3-a172-937ba70f8723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:16.913895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54600da1-fdfa-44b3-a172-937ba70f8723 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.914181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54600da1-fdfa-44b3-a172-937ba70f8723 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.914534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54600da1-fdfa-44b3-a172-937ba70f8723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.916952 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-54600da1-fdfa-44b3-a172-937ba70f8723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:16.919856 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a5f6b-8579-4522-b33d-8eba0059e076 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:16.929772 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54600da1-fdfa-44b3-a172-937ba70f8723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:16.931319 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54600da1-fdfa-44b3-a172-937ba70f8723 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:16.931764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332291b-c4d9-4a39-aeab-37ba86d3b7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:16.932638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332291b-c4d9-4a39-aeab-37ba86d3b7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.933070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332291b-c4d9-4a39-aeab-37ba86d3b7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.933497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332291b-c4d9-4a39-aeab-37ba86d3b7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.934199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c332291b-c4d9-4a39-aeab-37ba86d3b7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:16.936096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c332291b-c4d9-4a39-aeab-37ba86d3b7d9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:16.941003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 966/1925] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:16.942253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeebd60-dd22-405c-acf0-bdd92d30abb2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.942545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeebd60-dd22-405c-acf0-bdd92d30abb2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.942866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeebd60-dd22-405c-acf0-bdd92d30abb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.950735 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 960/1926] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:16 2022] GET /identity/v3/roles => generated 1427 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:16.953013 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332291b-c4d9-4a39-aeab-37ba86d3b7d9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:16.964454 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332291b-c4d9-4a39-aeab-37ba86d3b7d9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:16.967788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-beeebd60-dd22-405c-acf0-bdd92d30abb2 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:16.969007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-beeebd60-dd22-405c-acf0-bdd92d30abb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:16.969624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5f97b-a91e-4e87-b4a5-cc515195becf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:16.970326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5f97b-a91e-4e87-b4a5-cc515195becf admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.970698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5f97b-a91e-4e87-b4a5-cc515195becf admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.970982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5f97b-a91e-4e87-b4a5-cc515195becf admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.971733 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5f97b-a91e-4e87-b4a5-cc515195becf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:16.973338 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b5f97b-a91e-4e87-b4a5-cc515195becf admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:16.977432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 967/1927] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:16.987520 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 961/1928] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:16 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:15:16.987702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69525669-b776-4a30-ba4f-42a89a156ad1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:16.987702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69525669-b776-4a30-ba4f-42a89a156ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:16.987702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69525669-b776-4a30-ba4f-42a89a156ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:16.989199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5f97b-a91e-4e87-b4a5-cc515195becf admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:17.002736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b5f97b-a91e-4e87-b4a5-cc515195becf admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:17.007014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-69525669-b776-4a30-ba4f-42a89a156ad1 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:17.008238 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-69525669-b776-4a30-ba4f-42a89a156ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:17.010535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32a7c0-9ae1-4d73-97ae-c7ea223d4a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:17.011356 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32a7c0-9ae1-4d73-97ae-c7ea223d4a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:17.012090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32a7c0-9ae1-4d73-97ae-c7ea223d4a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:17.012422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32a7c0-9ae1-4d73-97ae-c7ea223d4a38 admin admin] PATH_INFO: `/v3/projects/371daf03994b41649d4edb279fab49ff/users/125e197e3bb549f7a113a91d0432f9df/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:17.022231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 968/1929] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:17.023478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69525669-b776-4a30-ba4f-42a89a156ad1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:17.028983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:17.034457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69525669-b776-4a30-ba4f-42a89a156ad1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:17.035529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32a7c0-9ae1-4d73-97ae-c7ea223d4a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=371daf03994b41649d4edb279fab49ff, user_id=125e197e3bb549f7a113a91d0432f9df, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:17.039985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32a7c0-9ae1-4d73-97ae-c7ea223d4a38 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:17.040136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87059d4e-d77a-4f9c-9ecc-7c4b16af2972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:17.040531 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87059d4e-d77a-4f9c-9ecc-7c4b16af2972 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:17.040786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87059d4e-d77a-4f9c-9ecc-7c4b16af2972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:17.041083 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87059d4e-d77a-4f9c-9ecc-7c4b16af2972 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:17.042091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87059d4e-d77a-4f9c-9ecc-7c4b16af2972 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:17.042450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87059d4e-d77a-4f9c-9ecc-7c4b16af2972 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:17.044204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87059d4e-d77a-4f9c-9ecc-7c4b16af2972 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:17.058708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 969/1930] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:15:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:15:17.062730 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87059d4e-d77a-4f9c-9ecc-7c4b16af2972 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:17.064575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 962/1931] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:16 2022] PUT /identity/v3/projects/371daf03994b41649d4edb279fab49ff/users/125e197e3bb549f7a113a91d0432f9df/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:17.075131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87059d4e-d77a-4f9c-9ecc-7c4b16af2972 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:17.076243 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dab2a5-62c5-4192-903f-76af4e0ff85c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:17.076746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dab2a5-62c5-4192-903f-76af4e0ff85c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:17.077435 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dab2a5-62c5-4192-903f-76af4e0ff85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:17.107895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-68dab2a5-62c5-4192-903f-76af4e0ff85c None None] MFA Rules not processed for user `125e197e3bb549f7a113a91d0432f9df`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:17.109472 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-68dab2a5-62c5-4192-903f-76af4e0ff85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:17.158352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd01fb-f2ff-4f2a-b39b-09a45826d420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:17.159313 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd01fb-f2ff-4f2a-b39b-09a45826d420 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:17.159733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd01fb-f2ff-4f2a-b39b-09a45826d420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:17.160143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd01fb-f2ff-4f2a-b39b-09a45826d420 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:17.161149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd01fb-f2ff-4f2a-b39b-09a45826d420 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:17.164078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd01fb-f2ff-4f2a-b39b-09a45826d420 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:17.184636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 963/1932] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:17.196263 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dab2a5-62c5-4192-903f-76af4e0ff85c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:17.198279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-68dab2a5-62c5-4192-903f-76af4e0ff85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:17.214410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 970/1933] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:17 2022] POST /identity/v3/projects => generated 378 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:17.218899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd01fb-f2ff-4f2a-b39b-09a45826d420 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:17.269466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dab2a5-62c5-4192-903f-76af4e0ff85c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:17.307581 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd01fb-f2ff-4f2a-b39b-09a45826d420 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:17.316041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7e613-43a2-4911-b1f4-1d47ce16c89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:17.317078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7e613-43a2-4911-b1f4-1d47ce16c89a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:17.317360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7e613-43a2-4911-b1f4-1d47ce16c89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:17.317638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7e613-43a2-4911-b1f4-1d47ce16c89a admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:17.318314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7e613-43a2-4911-b1f4-1d47ce16c89a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:17.320564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7e613-43a2-4911-b1f4-1d47ce16c89a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:17.329404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b229041-ff07-47fd-8568-93596defc147 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:17.330185 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b229041-ff07-47fd-8568-93596defc147 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:17.330339 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b229041-ff07-47fd-8568-93596defc147 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:17.330621 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b229041-ff07-47fd-8568-93596defc147 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:17.356395 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 971/1934] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:17 2022] POST /identity/v3/users => generated 290 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:17.366791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7e613-43a2-4911-b1f4-1d47ce16c89a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:17.394381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7e613-43a2-4911-b1f4-1d47ce16c89a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:17.399621 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ab5b7-f6b2-49cb-9121-bd866410f80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:17.400349 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ab5b7-f6b2-49cb-9121-bd866410f80b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:17.400636 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ab5b7-f6b2-49cb-9121-bd866410f80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:17.401128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5ab5b7-f6b2-49cb-9121-bd866410f80b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:17.402858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ab5b7-f6b2-49cb-9121-bd866410f80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:17.405630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5ab5b7-f6b2-49cb-9121-bd866410f80b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:17.408415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:17.416114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:17.435826 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 972/1935] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:17 2022] GET /identity/v3/roles => generated 1427 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:17.437169 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ab5b7-f6b2-49cb-9121-bd866410f80b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:17.437282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b229041-ff07-47fd-8568-93596defc147 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:17.438739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b229041-ff07-47fd-8568-93596defc147 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:17.458820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5ab5b7-f6b2-49cb-9121-bd866410f80b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:17.460533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 964/1936] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:17.469281 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f427548-f2b5-4f38-babc-8f81bbd9b667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:17.469997 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f427548-f2b5-4f38-babc-8f81bbd9b667 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:17.474971 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f427548-f2b5-4f38-babc-8f81bbd9b667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:17.475452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f427548-f2b5-4f38-babc-8f81bbd9b667 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:17.476756 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f427548-f2b5-4f38-babc-8f81bbd9b667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:17.479130 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f427548-f2b5-4f38-babc-8f81bbd9b667 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:17.527120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b229041-ff07-47fd-8568-93596defc147 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:17.529152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5b229041-ff07-47fd-8568-93596defc147 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:17.529963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 973/1937] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:17 2022] GET /identity/v3/roles => generated 1427 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:15:17.534384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f427548-f2b5-4f38-babc-8f81bbd9b667 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:17.551298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b229041-ff07-47fd-8568-93596defc147 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:17.566007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f427548-f2b5-4f38-babc-8f81bbd9b667 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:17.569894 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:17.573099 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:17.573613 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:17.573901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] PATH_INFO: `/v3/projects/99ca28da9fbb4a0499c1308445d03419` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:17.575601 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37918eb-c3d9-4c0d-9020-b0d8aeb269b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:17.576873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37918eb-c3d9-4c0d-9020-b0d8aeb269b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:17.577546 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37918eb-c3d9-4c0d-9020-b0d8aeb269b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:17.577837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37918eb-c3d9-4c0d-9020-b0d8aeb269b2 admin admin] PATH_INFO: `/v3/projects/c5a758f7a50e452a8ae7be20530159f5/users/13ef217efc554e30b008b4d96ab16527/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:17.578527 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ca28da9fbb4a0499c1308445d03419)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:17.586866 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:17.633795 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37918eb-c3d9-4c0d-9020-b0d8aeb269b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5a758f7a50e452a8ae7be20530159f5, user_id=13ef217efc554e30b008b4d96ab16527, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:17.646275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37918eb-c3d9-4c0d-9020-b0d8aeb269b2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:17.677029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 974/1938] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:17 2022] PUT /identity/v3/projects/c5a758f7a50e452a8ae7be20530159f5/users/13ef217efc554e30b008b4d96ab16527/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:17.684190 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ca28da9fbb4a0499c1308445d03419', 'request_id': 'req-6507c055-a3c3-4640-b119-1848a1835882'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:17.689128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd3a18-6469-48c0-b9be-49f30229a295 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:17.689551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd3a18-6469-48c0-b9be-49f30229a295 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:17.690009 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd3a18-6469-48c0-b9be-49f30229a295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:17.693347 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ca28da9fbb4a0499c1308445d03419', 'request_id': 'req-6507c055-a3c3-4640-b119-1848a1835882'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:17.705402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ca28da9fbb4a0499c1308445d03419', 'request_id': 'req-6507c055-a3c3-4640-b119-1848a1835882'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:17.725583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] The token cache is being invalidate because project 99ca28da9fbb4a0499c1308445d03419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:17.726687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:17.727682 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:17.731018 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 965/1939] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:17 2022] DELETE /identity/v3/projects/99ca28da9fbb4a0499c1308445d03419 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:17.734886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:17.736630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:17.776013 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1fdd3a18-6469-48c0-b9be-49f30229a295 None None] MFA Rules not processed for user `13ef217efc554e30b008b4d96ab16527`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:17.777474 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1fdd3a18-6469-48c0-b9be-49f30229a295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:17.842946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507c055-a3c3-4640-b119-1848a1835882 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:17.946057 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146bdb4-1f76-4880-9895-4d536655023d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:17.949147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146bdb4-1f76-4880-9895-4d536655023d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:17.949443 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146bdb4-1f76-4880-9895-4d536655023d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:17.949731 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146bdb4-1f76-4880-9895-4d536655023d admin admin] PATH_INFO: `/v3/users/25237b6239504d3fa20d7c02321cec55` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:17.951271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f146bdb4-1f76-4880-9895-4d536655023d admin admin] RBAC: Authorizing `identity:delete_user(user_id=25237b6239504d3fa20d7c02321cec55)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:17.957754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f146bdb4-1f76-4880-9895-4d536655023d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:17.980639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 975/1940] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:17.986632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd3a18-6469-48c0-b9be-49f30229a295 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:17.992380 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1fdd3a18-6469-48c0-b9be-49f30229a295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:18.022564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd3a18-6469-48c0-b9be-49f30229a295 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:18.024028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:18.052888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8a602-b847-45b0-b49e-71d14585a9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:18.053551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8a602-b847-45b0-b49e-71d14585a9e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:18.053839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8a602-b847-45b0-b49e-71d14585a9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:18.054121 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8a602-b847-45b0-b49e-71d14585a9e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:18.054944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8a602-b847-45b0-b49e-71d14585a9e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:18.057690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8a602-b847-45b0-b49e-71d14585a9e2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:18.073680 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f146bdb4-1f76-4880-9895-4d536655023d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25237b6239504d3fa20d7c02321cec55'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:18.086235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f146bdb4-1f76-4880-9895-4d536655023d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25237b6239504d3fa20d7c02321cec55'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:18.087370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f146bdb4-1f76-4880-9895-4d536655023d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25237b6239504d3fa20d7c02321cec55'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:18.088293 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f146bdb4-1f76-4880-9895-4d536655023d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25237b6239504d3fa20d7c02321cec55'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:18.093601 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 976/1941] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:17 2022] POST /identity/v3/projects => generated 380 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:18.098505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8a602-b847-45b0-b49e-71d14585a9e2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:18.100514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3fa8a602-b847-45b0-b49e-71d14585a9e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:18.116848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f146bdb4-1f76-4880-9895-4d536655023d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25237b6239504d3fa20d7c02321cec55'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:18.133245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 966/1942] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:17 2022] DELETE /identity/v3/users/25237b6239504d3fa20d7c02321cec55 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:18.226462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8a602-b847-45b0-b49e-71d14585a9e2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:18.267620 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:18.301090 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24730435-72a1-4e18-a0b8-e453194c4606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:18.301900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24730435-72a1-4e18-a0b8-e453194c4606 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:18.302187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24730435-72a1-4e18-a0b8-e453194c4606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:18.302490 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24730435-72a1-4e18-a0b8-e453194c4606 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:18.303208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24730435-72a1-4e18-a0b8-e453194c4606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:18.305273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24730435-72a1-4e18-a0b8-e453194c4606 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:18.336857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 977/1943] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:18 2022] POST /identity/v3/users => generated 297 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:18.338641 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146bdb4-1f76-4880-9895-4d536655023d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:18.357978 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146bdb4-1f76-4880-9895-4d536655023d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:18.370496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317dbb9a-4e19-4e93-ac3c-19fbfadea05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:18.373590 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dbb9a-4e19-4e93-ac3c-19fbfadea05f admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:18.373956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dbb9a-4e19-4e93-ac3c-19fbfadea05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:18.374421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dbb9a-4e19-4e93-ac3c-19fbfadea05f admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:18.376608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317dbb9a-4e19-4e93-ac3c-19fbfadea05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:18.378550 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317dbb9a-4e19-4e93-ac3c-19fbfadea05f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:18.394601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 967/1944] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:18 2022] GET /identity/v3/roles => generated 1427 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:18.398831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24730435-72a1-4e18-a0b8-e453194c4606 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:18.415000 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24730435-72a1-4e18-a0b8-e453194c4606 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:18.422322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24258641-a989-49a1-8f37-89f039642b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:18.423457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24258641-a989-49a1-8f37-89f039642b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:18.424002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24258641-a989-49a1-8f37-89f039642b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:18.424482 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24258641-a989-49a1-8f37-89f039642b51 admin admin] PATH_INFO: `/v3/projects/a748e92bd05a4757b6a87f53057777e4/users/3557e35ccd584cf5a2a7523cc1055ebe/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:18.452736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24258641-a989-49a1-8f37-89f039642b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a748e92bd05a4757b6a87f53057777e4, user_id=3557e35ccd584cf5a2a7523cc1055ebe, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:18.459644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24258641-a989-49a1-8f37-89f039642b51 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:18.499706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 978/1945] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:18 2022] PUT /identity/v3/projects/a748e92bd05a4757b6a87f53057777e4/users/3557e35ccd584cf5a2a7523cc1055ebe/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:18.503727 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317dbb9a-4e19-4e93-ac3c-19fbfadea05f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:18.519960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317dbb9a-4e19-4e93-ac3c-19fbfadea05f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:18.678965 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc872e1-860e-4f7b-b375-7d57590b92c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:18.680703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc872e1-860e-4f7b-b375-7d57590b92c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:18.681182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc872e1-860e-4f7b-b375-7d57590b92c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:18.681627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc872e1-860e-4f7b-b375-7d57590b92c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:18.683680 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc872e1-860e-4f7b-b375-7d57590b92c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:18.690207 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc872e1-860e-4f7b-b375-7d57590b92c2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:18.730735 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 968/1946] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:18 2022] GET /identity/v3/roles => generated 1427 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:15:18.737432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24258641-a989-49a1-8f37-89f039642b51 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:18.751023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24258641-a989-49a1-8f37-89f039642b51 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:18.757076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc1e36-61c1-4c71-8169-9d51ed3c56d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:18.757798 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc1e36-61c1-4c71-8169-9d51ed3c56d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:18.758086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc1e36-61c1-4c71-8169-9d51ed3c56d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:18.758370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc1e36-61c1-4c71-8169-9d51ed3c56d5 admin admin] PATH_INFO: `/v3/projects/a748e92bd05a4757b6a87f53057777e4/users/3557e35ccd584cf5a2a7523cc1055ebe/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:18.761887 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc1e36-61c1-4c71-8169-9d51ed3c56d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a748e92bd05a4757b6a87f53057777e4, user_id=3557e35ccd584cf5a2a7523cc1055ebe, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:18.764865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc1e36-61c1-4c71-8169-9d51ed3c56d5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:18.780991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 979/1947] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:18 2022] PUT /identity/v3/projects/a748e92bd05a4757b6a87f53057777e4/users/3557e35ccd584cf5a2a7523cc1055ebe/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:18.794093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782daf2-1e3b-4d68-8628-b4be2a0b7b18 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:18.794572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782daf2-1e3b-4d68-8628-b4be2a0b7b18 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:18.794914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c782daf2-1e3b-4d68-8628-b4be2a0b7b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:18.858495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c782daf2-1e3b-4d68-8628-b4be2a0b7b18 None None] MFA Rules not processed for user `3557e35ccd584cf5a2a7523cc1055ebe`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:18.860852 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c782daf2-1e3b-4d68-8628-b4be2a0b7b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:18.988847 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 969/1948] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:19.005262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc1e36-61c1-4c71-8169-9d51ed3c56d5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:19.010028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-75cc1e36-61c1-4c71-8169-9d51ed3c56d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:19.098988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc1e36-61c1-4c71-8169-9d51ed3c56d5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:19.102985 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee695ed-e969-4aca-b70c-2df029bab171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:19.104137 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee695ed-e969-4aca-b70c-2df029bab171 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:19.105257 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee695ed-e969-4aca-b70c-2df029bab171 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:19.105724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee695ed-e969-4aca-b70c-2df029bab171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:19.121682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee695ed-e969-4aca-b70c-2df029bab171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:19.123384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee695ed-e969-4aca-b70c-2df029bab171 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:19.137570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 980/1949] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:19.716901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782daf2-1e3b-4d68-8628-b4be2a0b7b18 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:19.719191 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c782daf2-1e3b-4d68-8628-b4be2a0b7b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:19.768878 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:19.859469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c782daf2-1e3b-4d68-8628-b4be2a0b7b18 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:19.866529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96768d-8426-4818-a8b2-91f3dc19455a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:19.867585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96768d-8426-4818-a8b2-91f3dc19455a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:19.867846 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96768d-8426-4818-a8b2-91f3dc19455a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:19.868198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96768d-8426-4818-a8b2-91f3dc19455a admin admin] PATH_INFO: `/v3/projects/240170495f714ffaaf4389d4471023b4` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:19.869771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96768d-8426-4818-a8b2-91f3dc19455a admin admin] RBAC: Authorizing `identity:delete_project(project_id=240170495f714ffaaf4389d4471023b4)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:19.872362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96768d-8426-4818-a8b2-91f3dc19455a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:19.955463 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-7a96768d-8426-4818-a8b2-91f3dc19455a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240170495f714ffaaf4389d4471023b4', 'request_id': 'req-7a96768d-8426-4818-a8b2-91f3dc19455a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:19.964036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-7a96768d-8426-4818-a8b2-91f3dc19455a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240170495f714ffaaf4389d4471023b4', 'request_id': 'req-7a96768d-8426-4818-a8b2-91f3dc19455a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:19.974923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee695ed-e969-4aca-b70c-2df029bab171 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:19.987473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee695ed-e969-4aca-b70c-2df029bab171 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:19.989290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-7a96768d-8426-4818-a8b2-91f3dc19455a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240170495f714ffaaf4389d4471023b4', 'request_id': 'req-7a96768d-8426-4818-a8b2-91f3dc19455a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.006893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-7a96768d-8426-4818-a8b2-91f3dc19455a admin admin] The token cache is being invalidate because project 240170495f714ffaaf4389d4471023b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:20.007114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-7a96768d-8426-4818-a8b2-91f3dc19455a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.008148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-7a96768d-8426-4818-a8b2-91f3dc19455a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.010097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 970/1950] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:19 2022] DELETE /identity/v3/projects/240170495f714ffaaf4389d4471023b4 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:20.016541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e441590-c18f-44d4-ab25-66fac843c394 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.016921 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e441590-c18f-44d4-ab25-66fac843c394 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.017259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e441590-c18f-44d4-ab25-66fac843c394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.046359 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5e441590-c18f-44d4-ab25-66fac843c394 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:20.047645 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5e441590-c18f-44d4-ab25-66fac843c394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:20.085448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c74cdd-d5b3-4e11-bfa1-1490076b0d2a tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3557e35ccd584cf5a2a7523cc1055ebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a748e92bd05a4757b6a87f53057777e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:20.086080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c74cdd-d5b3-4e11-bfa1-1490076b0d2a tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.086396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c74cdd-d5b3-4e11-bfa1-1490076b0d2a tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.086693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c74cdd-d5b3-4e11-bfa1-1490076b0d2a tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.088001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c74cdd-d5b3-4e11-bfa1-1490076b0d2a tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:20.088311 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c74cdd-d5b3-4e11-bfa1-1490076b0d2a tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:20.089963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c74cdd-d5b3-4e11-bfa1-1490076b0d2a tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:20.102042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 981/1951] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:15:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:15:20.107207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c74cdd-d5b3-4e11-bfa1-1490076b0d2a tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:20.109432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a8c74cdd-d5b3-4e11-bfa1-1490076b0d2a tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:20.121490 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 971/1952] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:20.122535 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c74cdd-d5b3-4e11-bfa1-1490076b0d2a tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:20.127148 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ab315-48be-4741-b66c-743f8e1781a4 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3557e35ccd584cf5a2a7523cc1055ebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a748e92bd05a4757b6a87f53057777e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:20.127770 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ab315-48be-4741-b66c-743f8e1781a4 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.128054 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ab315-48be-4741-b66c-743f8e1781a4 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.128332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ab315-48be-4741-b66c-743f8e1781a4 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.129001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db997aa9-0749-486a-93f4-f49ea9cee9d9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.129122 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ab315-48be-4741-b66c-743f8e1781a4 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:20.129298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db997aa9-0749-486a-93f4-f49ea9cee9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.129579 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db997aa9-0749-486a-93f4-f49ea9cee9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.131190 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ab315-48be-4741-b66c-743f8e1781a4 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:20.152508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-db997aa9-0749-486a-93f4-f49ea9cee9d9 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:20.152982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-db997aa9-0749-486a-93f4-f49ea9cee9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:20.165091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 972/1953] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:20.170347 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db997aa9-0749-486a-93f4-f49ea9cee9d9 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:20.171833 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 982/1954] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:20 2022] POST /identity/v3/projects => generated 354 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:20.172743 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ab315-48be-4741-b66c-743f8e1781a4 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:20.179174 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:20.180412 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db997aa9-0749-486a-93f4-f49ea9cee9d9 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:20.217307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:20.247608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d731f09-bb22-4169-a499-78d89c643870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:20.248103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ab315-48be-4741-b66c-743f8e1781a4 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:20.248240 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d731f09-bb22-4169-a499-78d89c643870 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.248524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d731f09-bb22-4169-a499-78d89c643870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.248802 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d731f09-bb22-4169-a499-78d89c643870 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.249502 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d731f09-bb22-4169-a499-78d89c643870 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:20.249808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d731f09-bb22-4169-a499-78d89c643870 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:20.251449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d731f09-bb22-4169-a499-78d89c643870 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:20.253074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecb19d-75be-4a51-b89b-2d9491ba5911 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3557e35ccd584cf5a2a7523cc1055ebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a748e92bd05a4757b6a87f53057777e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:20.253696 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecb19d-75be-4a51-b89b-2d9491ba5911 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.253977 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecb19d-75be-4a51-b89b-2d9491ba5911 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.254256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecb19d-75be-4a51-b89b-2d9491ba5911 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.254980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cecb19d-75be-4a51-b89b-2d9491ba5911 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:20.255287 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cecb19d-75be-4a51-b89b-2d9491ba5911 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:20.256860 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cecb19d-75be-4a51-b89b-2d9491ba5911 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:20.264186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 973/1955] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:15:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:15:20.268787 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d731f09-bb22-4169-a499-78d89c643870 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:20.272999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 983/1956] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:15:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:15:20.279848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d731f09-bb22-4169-a499-78d89c643870 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:20.284547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bfefe-a48e-4b69-b0b1-49caed614d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:20.285163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bfefe-a48e-4b69-b0b1-49caed614d59 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.285450 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bfefe-a48e-4b69-b0b1-49caed614d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.285726 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bfefe-a48e-4b69-b0b1-49caed614d59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.286395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4bfefe-a48e-4b69-b0b1-49caed614d59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:20.288332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4bfefe-a48e-4b69-b0b1-49caed614d59 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:20.309910 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecb19d-75be-4a51-b89b-2d9491ba5911 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:20.316560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 974/1957] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:20 2022] POST /identity/v3/projects => generated 388 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:20.320384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bfefe-a48e-4b69-b0b1-49caed614d59 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:20.320809 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecb19d-75be-4a51-b89b-2d9491ba5911 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:20.381198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3557e35ccd584cf5a2a7523cc1055ebe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a748e92bd05a4757b6a87f53057777e4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:20.382047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.382337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.382626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] PATH_INFO: `/v3/projects/a001564719e1448f99b219367bf7c0bb` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.384354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a001564719e1448f99b219367bf7c0bb)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:20.385440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bfefe-a48e-4b69-b0b1-49caed614d59 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:20.386272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:20.390389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8ec23-f98e-44a2-824a-74d5fe27b9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:20.391113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8ec23-f98e-44a2-824a-74d5fe27b9c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.391396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8ec23-f98e-44a2-824a-74d5fe27b9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.391671 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8ec23-f98e-44a2-824a-74d5fe27b9c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.392372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8ec23-f98e-44a2-824a-74d5fe27b9c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:20.394318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8ec23-f98e-44a2-824a-74d5fe27b9c6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:20.421379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 975/1958] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:20 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:20.422899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8ec23-f98e-44a2-824a-74d5fe27b9c6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:20.434061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8ec23-f98e-44a2-824a-74d5fe27b9c6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:20.448217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a001564719e1448f99b219367bf7c0bb', 'request_id': 'req-185206a3-b73f-44ea-bdf9-7ba8163bb645'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.456610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a001564719e1448f99b219367bf7c0bb', 'request_id': 'req-185206a3-b73f-44ea-bdf9-7ba8163bb645'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.467188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a001564719e1448f99b219367bf7c0bb', 'request_id': 'req-185206a3-b73f-44ea-bdf9-7ba8163bb645'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.485367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] The token cache is being invalidate because project a001564719e1448f99b219367bf7c0bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:20.485653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.486331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.491483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 984/1959] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:20 2022] DELETE /identity/v3/projects/a001564719e1448f99b219367bf7c0bb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:20.518606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb160e-0524-4a03-a9b9-d3a2473132fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:20.519570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb160e-0524-4a03-a9b9-d3a2473132fc admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.520032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb160e-0524-4a03-a9b9-d3a2473132fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.520485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb160e-0524-4a03-a9b9-d3a2473132fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.521576 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb160e-0524-4a03-a9b9-d3a2473132fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:20.523577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb160e-0524-4a03-a9b9-d3a2473132fc admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:20.546057 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 976/1960] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:20 2022] GET /identity/v3/roles => generated 1427 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:15:20.552993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:20.556890 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:20.573866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185206a3-b73f-44ea-bdf9-7ba8163bb645 tempest-VolumeQuotaClassesTest-703787192 tempest-VolumeQuotaClassesTest-703787192-project-admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:20.580587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce53f0-bf27-4f4f-bf3e-829b0118aefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:20.581497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce53f0-bf27-4f4f-bf3e-829b0118aefc admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.581898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce53f0-bf27-4f4f-bf3e-829b0118aefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.583281 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce53f0-bf27-4f4f-bf3e-829b0118aefc admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.584249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce53f0-bf27-4f4f-bf3e-829b0118aefc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:20.588081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce53f0-bf27-4f4f-bf3e-829b0118aefc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:20.606425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 985/1961] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:20 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:20.608366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb160e-0524-4a03-a9b9-d3a2473132fc admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:20.623585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb160e-0524-4a03-a9b9-d3a2473132fc admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:20.632070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc9ae-5c97-47b1-a347-51a603516879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:20.633355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc9ae-5c97-47b1-a347-51a603516879 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.633686 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc9ae-5c97-47b1-a347-51a603516879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.634637 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc9ae-5c97-47b1-a347-51a603516879 admin admin] PATH_INFO: `/v3/projects/5d713d6c91a34883a130ff954257c706/users/4072bff1da7142ce963fce6a558d0d6b/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.659184 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fc9ae-5c97-47b1-a347-51a603516879 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d713d6c91a34883a130ff954257c706, user_id=4072bff1da7142ce963fce6a558d0d6b, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:20.663601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fc9ae-5c97-47b1-a347-51a603516879 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:20.697595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 977/1962] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:20 2022] PUT /identity/v3/projects/5d713d6c91a34883a130ff954257c706/users/4072bff1da7142ce963fce6a558d0d6b/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:20.700204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f621f-39b3-45d0-a370-53216884b828 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.700469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f621f-39b3-45d0-a370-53216884b828 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.714688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f621f-39b3-45d0-a370-53216884b828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.741441 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-528f621f-39b3-45d0-a370-53216884b828 None None] MFA Rules not processed for user `4072bff1da7142ce963fce6a558d0d6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:20.742689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-528f621f-39b3-45d0-a370-53216884b828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:20.749455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:20.778231 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc9ae-5c97-47b1-a347-51a603516879 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:20.780610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ce4fc9ae-5c97-47b1-a347-51a603516879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:20.793408 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:20.822259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 986/1963] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:20.826841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f621f-39b3-45d0-a370-53216884b828 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:20.853568 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc9ae-5c97-47b1-a347-51a603516879 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:20.858606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df663fc7-e080-4019-94cb-3f13a5063f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:20.859323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df663fc7-e080-4019-94cb-3f13a5063f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.859622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df663fc7-e080-4019-94cb-3f13a5063f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.859939 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df663fc7-e080-4019-94cb-3f13a5063f33 admin admin] PATH_INFO: `/v3/users/13ef217efc554e30b008b4d96ab16527` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.861177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df663fc7-e080-4019-94cb-3f13a5063f33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ef217efc554e30b008b4d96ab16527)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:20.863146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df663fc7-e080-4019-94cb-3f13a5063f33 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:20.890882 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f621f-39b3-45d0-a370-53216884b828 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:20.895829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8e88e-a137-4b9c-af73-d01e407ee15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:20.896519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8e88e-a137-4b9c-af73-d01e407ee15c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.896808 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8e88e-a137-4b9c-af73-d01e407ee15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.897089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8e88e-a137-4b9c-af73-d01e407ee15c admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.897816 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8e88e-a137-4b9c-af73-d01e407ee15c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:20.900052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8e88e-a137-4b9c-af73-d01e407ee15c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:20.911515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-df663fc7-e080-4019-94cb-3f13a5063f33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ef217efc554e30b008b4d96ab16527'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.922804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-df663fc7-e080-4019-94cb-3f13a5063f33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ef217efc554e30b008b4d96ab16527'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.923468 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-df663fc7-e080-4019-94cb-3f13a5063f33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ef217efc554e30b008b4d96ab16527'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.924503 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-df663fc7-e080-4019-94cb-3f13a5063f33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ef217efc554e30b008b4d96ab16527'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.930300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 987/1964] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:20 2022] POST /identity/v3/projects => generated 388 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:20.934569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8e88e-a137-4b9c-af73-d01e407ee15c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:20.936337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b7e8e88e-a137-4b9c-af73-d01e407ee15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:20.940443 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-df663fc7-e080-4019-94cb-3f13a5063f33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ef217efc554e30b008b4d96ab16527'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:20.964230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 978/1965] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:20 2022] DELETE /identity/v3/users/13ef217efc554e30b008b4d96ab16527 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:20.969501 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfddb61-0c13-4691-995c-a2674ba14e16 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:20.969918 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfddb61-0c13-4691-995c-a2674ba14e16 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:20.970383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfddb61-0c13-4691-995c-a2674ba14e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:20.990810 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-8cfddb61-0c13-4691-995c-a2674ba14e16 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:20.992170 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8cfddb61-0c13-4691-995c-a2674ba14e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:21.018082 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8e88e-a137-4b9c-af73-d01e407ee15c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:21.055026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:21.062319 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 979/1966] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:21.073372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfddb61-0c13-4691-995c-a2674ba14e16 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:21.074950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8cfddb61-0c13-4691-995c-a2674ba14e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:21.095080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57f57b-b255-455a-9135-41b2f1847029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:21.096162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57f57b-b255-455a-9135-41b2f1847029 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:21.096442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57f57b-b255-455a-9135-41b2f1847029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:21.096771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57f57b-b255-455a-9135-41b2f1847029 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:21.097858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57f57b-b255-455a-9135-41b2f1847029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:21.100433 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57f57b-b255-455a-9135-41b2f1847029 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:21.118267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:21.138776 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 988/1967] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:20 2022] POST /identity/v3/users => generated 301 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:21.143224 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57f57b-b255-455a-9135-41b2f1847029 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:21.155336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfddb61-0c13-4691-995c-a2674ba14e16 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:21.156747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57f57b-b255-455a-9135-41b2f1847029 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:21.159406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef95e44-09b7-4d0f-86a4-91c4b4675aa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:21.159646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef95e44-09b7-4d0f-86a4-91c4b4675aa4 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:21.160097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef95e44-09b7-4d0f-86a4-91c4b4675aa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:21.160375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef95e44-09b7-4d0f-86a4-91c4b4675aa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:21.163014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec81f5-8c21-43c5-aacb-15210152e50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:21.163702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec81f5-8c21-43c5-aacb-15210152e50d admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:21.164103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec81f5-8c21-43c5-aacb-15210152e50d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:21.164446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec81f5-8c21-43c5-aacb-15210152e50d admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:21.165733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec81f5-8c21-43c5-aacb-15210152e50d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:21.167489 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec81f5-8c21-43c5-aacb-15210152e50d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:21.171941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef95e44-09b7-4d0f-86a4-91c4b4675aa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:21.173112 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef95e44-09b7-4d0f-86a4-91c4b4675aa4 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:21.181509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 989/1968] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:21 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:21.183717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 980/1969] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:21.185482 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec81f5-8c21-43c5-aacb-15210152e50d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:21.196933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec81f5-8c21-43c5-aacb-15210152e50d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:21.207912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255d6fc-d995-42c0-936a-afcaaff92486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:21.208614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255d6fc-d995-42c0-936a-afcaaff92486 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:21.208896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255d6fc-d995-42c0-936a-afcaaff92486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:21.209172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255d6fc-d995-42c0-936a-afcaaff92486 admin admin] PATH_INFO: `/v3/projects/be3208c8758348a1af50872006a120cf/users/5e3fe38317c94c548147ee14f247c08a/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:21.230728 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c255d6fc-d995-42c0-936a-afcaaff92486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3208c8758348a1af50872006a120cf, user_id=5e3fe38317c94c548147ee14f247c08a, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:21.233442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c255d6fc-d995-42c0-936a-afcaaff92486 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:21.254042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 990/1970] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:21 2022] PUT /identity/v3/projects/be3208c8758348a1af50872006a120cf/users/5e3fe38317c94c548147ee14f247c08a/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:21.259898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef95e44-09b7-4d0f-86a4-91c4b4675aa4 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:21.270950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef95e44-09b7-4d0f-86a4-91c4b4675aa4 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:21.331272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30e73d-951a-4b3b-9de6-9b9bdabe7bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:21.331901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30e73d-951a-4b3b-9de6-9b9bdabe7bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:21.332451 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30e73d-951a-4b3b-9de6-9b9bdabe7bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:21.332736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30e73d-951a-4b3b-9de6-9b9bdabe7bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:21.333436 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d30e73d-951a-4b3b-9de6-9b9bdabe7bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:21.335123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d30e73d-951a-4b3b-9de6-9b9bdabe7bb2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:21.351761 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 981/1971] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:21 2022] GET /identity/v3/roles => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:21.353554 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255d6fc-d995-42c0-936a-afcaaff92486 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:21.364081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255d6fc-d995-42c0-936a-afcaaff92486 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:21.369234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1479d-4f18-42e1-b926-794096019fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:21.369945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1479d-4f18-42e1-b926-794096019fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:21.370233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1479d-4f18-42e1-b926-794096019fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:21.370596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1479d-4f18-42e1-b926-794096019fc9 admin admin] PATH_INFO: `/v3/projects/be3208c8758348a1af50872006a120cf/users/5e3fe38317c94c548147ee14f247c08a/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:21.373134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1479d-4f18-42e1-b926-794096019fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3208c8758348a1af50872006a120cf, user_id=5e3fe38317c94c548147ee14f247c08a, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:21.376626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1479d-4f18-42e1-b926-794096019fc9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:21.387656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 991/1972] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:21 2022] PUT /identity/v3/projects/be3208c8758348a1af50872006a120cf/users/5e3fe38317c94c548147ee14f247c08a/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:21.398595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b761a9-6c3d-4255-9b09-0e77b629812e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:21.398933 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b761a9-6c3d-4255-9b09-0e77b629812e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:21.399244 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b761a9-6c3d-4255-9b09-0e77b629812e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:21.429348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-12b761a9-6c3d-4255-9b09-0e77b629812e None None] MFA Rules not processed for user `5e3fe38317c94c548147ee14f247c08a`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:21.430569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-12b761a9-6c3d-4255-9b09-0e77b629812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:21.480124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1479d-4f18-42e1-b926-794096019fc9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:21.482176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e4d1479d-4f18-42e1-b926-794096019fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:21.513340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 982/1973] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:21.524279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b761a9-6c3d-4255-9b09-0e77b629812e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:21.526334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-12b761a9-6c3d-4255-9b09-0e77b629812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:21.567766 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1479d-4f18-42e1-b926-794096019fc9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:21.572692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:21.573342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:21.573661 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:21.573904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] PATH_INFO: `/v3/projects/c5a758f7a50e452a8ae7be20530159f5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:21.575100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5a758f7a50e452a8ae7be20530159f5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:21.577010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:21.592697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b761a9-6c3d-4255-9b09-0e77b629812e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:21.596382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923c043-dab1-4a20-bda8-9564eae699ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:21.596996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923c043-dab1-4a20-bda8-9564eae699ce service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:21.597276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923c043-dab1-4a20-bda8-9564eae699ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:21.597696 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923c043-dab1-4a20-bda8-9564eae699ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:21.604226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:21.610259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923c043-dab1-4a20-bda8-9564eae699ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:21.610608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923c043-dab1-4a20-bda8-9564eae699ce service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:21.622134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 983/1974] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:21 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:21.627670 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5a758f7a50e452a8ae7be20530159f5', 'request_id': 'req-a9190fc6-4ee8-48c2-b700-151cd1695348'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.635060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5a758f7a50e452a8ae7be20530159f5', 'request_id': 'req-a9190fc6-4ee8-48c2-b700-151cd1695348'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.641173 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5a758f7a50e452a8ae7be20530159f5', 'request_id': 'req-a9190fc6-4ee8-48c2-b700-151cd1695348'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.655209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] The token cache is being invalidate because project c5a758f7a50e452a8ae7be20530159f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:21.655529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.656176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.660038 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 992/1975] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:21 2022] DELETE /identity/v3/projects/c5a758f7a50e452a8ae7be20530159f5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:21.664062 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923c043-dab1-4a20-bda8-9564eae699ce service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:21.675630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923c043-dab1-4a20-bda8-9564eae699ce service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:21.742789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b350c0-9052-4c7e-8d30-451f1d15e0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:21.744323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b350c0-9052-4c7e-8d30-451f1d15e0cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:21.744689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b350c0-9052-4c7e-8d30-451f1d15e0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:21.762031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b350c0-9052-4c7e-8d30-451f1d15e0cb admin admin] PATH_INFO: `/v3/users/3557e35ccd584cf5a2a7523cc1055ebe` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:21.764947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b350c0-9052-4c7e-8d30-451f1d15e0cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3557e35ccd584cf5a2a7523cc1055ebe)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:21.768662 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b350c0-9052-4c7e-8d30-451f1d15e0cb admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:21.833214 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a7b350c0-9052-4c7e-8d30-451f1d15e0cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3557e35ccd584cf5a2a7523cc1055ebe'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.841122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a7b350c0-9052-4c7e-8d30-451f1d15e0cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3557e35ccd584cf5a2a7523cc1055ebe'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.841838 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a7b350c0-9052-4c7e-8d30-451f1d15e0cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3557e35ccd584cf5a2a7523cc1055ebe'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.842746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a7b350c0-9052-4c7e-8d30-451f1d15e0cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3557e35ccd584cf5a2a7523cc1055ebe'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.861939 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a7b350c0-9052-4c7e-8d30-451f1d15e0cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3557e35ccd584cf5a2a7523cc1055ebe'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:21.881520 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 984/1976] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:21 2022] DELETE /identity/v3/users/3557e35ccd584cf5a2a7523cc1055ebe => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:22.237418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:22.239225 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:22.255932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9190fc6-4ee8-48c2-b700-151cd1695348 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:22.288323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:22.296422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b350c0-9052-4c7e-8d30-451f1d15e0cb admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:22.298097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a7b350c0-9052-4c7e-8d30-451f1d15e0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:22.378831 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b350c0-9052-4c7e-8d30-451f1d15e0cb admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:22.379184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbac64-f323-44a3-bb95-596e2e9bb298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:22.379966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbac64-f323-44a3-bb95-596e2e9bb298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.380319 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbac64-f323-44a3-bb95-596e2e9bb298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.380724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbac64-f323-44a3-bb95-596e2e9bb298 admin admin] PATH_INFO: `/v3/projects/a748e92bd05a4757b6a87f53057777e4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.382087 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbac64-f323-44a3-bb95-596e2e9bb298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a748e92bd05a4757b6a87f53057777e4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:22.384763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbac64-f323-44a3-bb95-596e2e9bb298 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:22.384925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4663e1-434f-4461-ade2-fd8bb10fd6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:22.385680 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4663e1-434f-4461-ade2-fd8bb10fd6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.386038 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4663e1-434f-4461-ade2-fd8bb10fd6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.386649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4663e1-434f-4461-ade2-fd8bb10fd6f5 admin admin] PATH_INFO: `/v3/users/4072bff1da7142ce963fce6a558d0d6b` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.387975 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4663e1-434f-4461-ade2-fd8bb10fd6f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4072bff1da7142ce963fce6a558d0d6b)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:22.390315 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4663e1-434f-4461-ade2-fd8bb10fd6f5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:22.406780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:22.448203 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ebbac64-f323-44a3-bb95-596e2e9bb298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a748e92bd05a4757b6a87f53057777e4', 'request_id': 'req-5ebbac64-f323-44a3-bb95-596e2e9bb298'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:22.450965 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ad4663e1-434f-4461-ade2-fd8bb10fd6f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4072bff1da7142ce963fce6a558d0d6b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:22.459782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ebbac64-f323-44a3-bb95-596e2e9bb298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a748e92bd05a4757b6a87f53057777e4', 'request_id': 'req-5ebbac64-f323-44a3-bb95-596e2e9bb298'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:22.461091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ad4663e1-434f-4461-ade2-fd8bb10fd6f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4072bff1da7142ce963fce6a558d0d6b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:22.462350 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ad4663e1-434f-4461-ade2-fd8bb10fd6f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4072bff1da7142ce963fce6a558d0d6b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:22.463396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ad4663e1-434f-4461-ade2-fd8bb10fd6f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4072bff1da7142ce963fce6a558d0d6b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:22.466684 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ebbac64-f323-44a3-bb95-596e2e9bb298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a748e92bd05a4757b6a87f53057777e4', 'request_id': 'req-5ebbac64-f323-44a3-bb95-596e2e9bb298'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:22.478941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ad4663e1-434f-4461-ade2-fd8bb10fd6f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4072bff1da7142ce963fce6a558d0d6b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:22.481913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ebbac64-f323-44a3-bb95-596e2e9bb298 admin admin] The token cache is being invalidate because project a748e92bd05a4757b6a87f53057777e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:22.482208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ebbac64-f323-44a3-bb95-596e2e9bb298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:22.483119 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ebbac64-f323-44a3-bb95-596e2e9bb298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:22.484875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 993/1977] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:22 2022] DELETE /identity/v3/projects/a748e92bd05a4757b6a87f53057777e4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:22.489595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58673144-2d93-4d40-88b0-546b9547cb07 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.489885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58673144-2d93-4d40-88b0-546b9547cb07 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.490167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58673144-2d93-4d40-88b0-546b9547cb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.492198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 985/1978] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:22 2022] DELETE /identity/v3/users/4072bff1da7142ce963fce6a558d0d6b => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:22.497109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f1d22-d5e1-4975-881b-a669b09cd3fd None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.497399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f1d22-d5e1-4975-881b-a669b09cd3fd None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.497680 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f1d22-d5e1-4975-881b-a669b09cd3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.518783 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-639f1d22-d5e1-4975-881b-a669b09cd3fd None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:22.519581 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-58673144-2d93-4d40-88b0-546b9547cb07 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:22.520046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-639f1d22-d5e1-4975-881b-a669b09cd3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:22.521553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-58673144-2d93-4d40-88b0-546b9547cb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:22.589013 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 986/1979] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:22.599096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f1d22-d5e1-4975-881b-a669b09cd3fd None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:22.599916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 994/1980] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:22.600639 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-639f1d22-d5e1-4975-881b-a669b09cd3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:22.608929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b685e89-e59e-404f-94c0-03def1f2aaf2 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.609220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b685e89-e59e-404f-94c0-03def1f2aaf2 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.609506 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b685e89-e59e-404f-94c0-03def1f2aaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.629354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2b685e89-e59e-404f-94c0-03def1f2aaf2 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:22.630690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2b685e89-e59e-404f-94c0-03def1f2aaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:22.640501 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 995/1981] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:22.647899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b685e89-e59e-404f-94c0-03def1f2aaf2 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:22.658524 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b685e89-e59e-404f-94c0-03def1f2aaf2 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:22.661876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f1d22-d5e1-4975-881b-a669b09cd3fd None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:22.664008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03150858-0337-46ff-8bdd-6a7b63ff80c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:22.664660 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03150858-0337-46ff-8bdd-6a7b63ff80c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.664940 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03150858-0337-46ff-8bdd-6a7b63ff80c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.665759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03150858-0337-46ff-8bdd-6a7b63ff80c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.666488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03150858-0337-46ff-8bdd-6a7b63ff80c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:22.666614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2a0d9-b8c6-4d7e-9695-6104fe178858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:22.666829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03150858-0337-46ff-8bdd-6a7b63ff80c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:22.667401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2a0d9-b8c6-4d7e-9695-6104fe178858 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.667691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2a0d9-b8c6-4d7e-9695-6104fe178858 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.667966 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2a0d9-b8c6-4d7e-9695-6104fe178858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.668434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03150858-0337-46ff-8bdd-6a7b63ff80c7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:22.677946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2a0d9-b8c6-4d7e-9695-6104fe178858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:22.679490 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2a0d9-b8c6-4d7e-9695-6104fe178858 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:22.681862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 996/1982] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:15:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:15:22.686145 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03150858-0337-46ff-8bdd-6a7b63ff80c7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:22.690296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 987/1983] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:22.697916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03150858-0337-46ff-8bdd-6a7b63ff80c7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:22.702709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4deb39-681f-4577-b67a-bb9515ca0684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:22.703352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4deb39-681f-4577-b67a-bb9515ca0684 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.703644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4deb39-681f-4577-b67a-bb9515ca0684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.703917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4deb39-681f-4577-b67a-bb9515ca0684 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.704597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4deb39-681f-4577-b67a-bb9515ca0684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:22.706514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4deb39-681f-4577-b67a-bb9515ca0684 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:22.730220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:22.739522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 997/1984] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:22 2022] POST /identity/v3/projects => generated 384 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:22.744462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2a0d9-b8c6-4d7e-9695-6104fe178858 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:22.756697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2a0d9-b8c6-4d7e-9695-6104fe178858 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:22.791331 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:22.821917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883c6d4-be37-4e67-bba1-a78384892767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:22.822646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883c6d4-be37-4e67-bba1-a78384892767 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.823016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883c6d4-be37-4e67-bba1-a78384892767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.823298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883c6d4-be37-4e67-bba1-a78384892767 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.823985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883c6d4-be37-4e67-bba1-a78384892767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:22.825943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883c6d4-be37-4e67-bba1-a78384892767 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:22.859001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 988/1985] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:22 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:22.862831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4deb39-681f-4577-b67a-bb9515ca0684 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:22.877429 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4deb39-681f-4577-b67a-bb9515ca0684 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:22.882483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b898d-fded-41dd-ac97-a6b402e291f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:22.883162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b898d-fded-41dd-ac97-a6b402e291f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.883593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b898d-fded-41dd-ac97-a6b402e291f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.883896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b898d-fded-41dd-ac97-a6b402e291f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.884594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b898d-fded-41dd-ac97-a6b402e291f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:22.886226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b898d-fded-41dd-ac97-a6b402e291f7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:22.900616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 998/1986] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:22 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:22.901899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883c6d4-be37-4e67-bba1-a78384892767 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:22.912747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883c6d4-be37-4e67-bba1-a78384892767 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:22.918206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbeca3-5a2b-4718-91c0-445999733c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:22.919049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbeca3-5a2b-4718-91c0-445999733c35 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.919330 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbeca3-5a2b-4718-91c0-445999733c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.919605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbeca3-5a2b-4718-91c0-445999733c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.920323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbeca3-5a2b-4718-91c0-445999733c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:22.922649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbeca3-5a2b-4718-91c0-445999733c35 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:22.936937 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 989/1987] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:22 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:22.938135 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b898d-fded-41dd-ac97-a6b402e291f7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:22.948937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b898d-fded-41dd-ac97-a6b402e291f7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:22.954544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1722a8b-871d-4b66-9fc4-4f44a6f62825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:22.955304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1722a8b-871d-4b66-9fc4-4f44a6f62825 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:22.955585 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1722a8b-871d-4b66-9fc4-4f44a6f62825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:22.955862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1722a8b-871d-4b66-9fc4-4f44a6f62825 admin admin] PATH_INFO: `/v3/projects/bd7e1c1ac7c64876a9227bfff6aaf4c8/users/f5fb2a47a44942eba2237b15e1c1f4c9/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:22.973245 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1722a8b-871d-4b66-9fc4-4f44a6f62825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd7e1c1ac7c64876a9227bfff6aaf4c8, user_id=f5fb2a47a44942eba2237b15e1c1f4c9, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:22.976094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1722a8b-871d-4b66-9fc4-4f44a6f62825 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:22.988977 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbeca3-5a2b-4718-91c0-445999733c35 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:22.991388 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-15bbeca3-5a2b-4718-91c0-445999733c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:22.997831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 999/1988] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:22 2022] PUT /identity/v3/projects/bd7e1c1ac7c64876a9227bfff6aaf4c8/users/f5fb2a47a44942eba2237b15e1c1f4c9/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:23.014969 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd668a-e2b1-4224-9cc3-10969f027385 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.015333 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd668a-e2b1-4224-9cc3-10969f027385 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.015516 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd668a-e2b1-4224-9cc3-10969f027385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.029036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbeca3-5a2b-4718-91c0-445999733c35 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:23.054511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-54bd668a-e2b1-4224-9cc3-10969f027385 None None] MFA Rules not processed for user `f5fb2a47a44942eba2237b15e1c1f4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:23.055867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-54bd668a-e2b1-4224-9cc3-10969f027385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:23.106816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:23.107477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.107752 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.108032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] PATH_INFO: `/v3/projects/5d713d6c91a34883a130ff954257c706` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.109267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d713d6c91a34883a130ff954257c706)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:23.111265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:23.148071 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1000/1989] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:23.155748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd668a-e2b1-4224-9cc3-10969f027385 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:23.170017 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd668a-e2b1-4224-9cc3-10969f027385 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:23.188311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d713d6c91a34883a130ff954257c706', 'request_id': 'req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.198501 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d713d6c91a34883a130ff954257c706', 'request_id': 'req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.214291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d713d6c91a34883a130ff954257c706', 'request_id': 'req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.234131 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] The token cache is being invalidate because project 5d713d6c91a34883a130ff954257c706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:23.234441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.235223 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.237184 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 990/1990] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:22 2022] DELETE /identity/v3/projects/5d713d6c91a34883a130ff954257c706 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:23.241160 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:23.243410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:23.248303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cf674-94eb-43a6-8fd0-1ea40cd73353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:23.249025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cf674-94eb-43a6-8fd0-1ea40cd73353 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.249281 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cf674-94eb-43a6-8fd0-1ea40cd73353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.249587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cf674-94eb-43a6-8fd0-1ea40cd73353 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.250283 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26cf674-94eb-43a6-8fd0-1ea40cd73353 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:23.252281 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26cf674-94eb-43a6-8fd0-1ea40cd73353 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:23.258079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cabe4a8-ff3c-44a9-88cb-7bce9e664dc9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:23.267998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efe230-124c-4612-a6c8-4468372630ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:23.268918 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efe230-124c-4612-a6c8-4468372630ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.269360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efe230-124c-4612-a6c8-4468372630ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.269760 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61efe230-124c-4612-a6c8-4468372630ad admin admin] PATH_INFO: `/v3/users/5e3fe38317c94c548147ee14f247c08a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.271334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61efe230-124c-4612-a6c8-4468372630ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e3fe38317c94c548147ee14f247c08a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:23.273521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61efe230-124c-4612-a6c8-4468372630ad admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:23.291300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1001/1991] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:23 2022] POST /identity/v3/projects => generated 384 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:23.292440 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cf674-94eb-43a6-8fd0-1ea40cd73353 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:23.294695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d26cf674-94eb-43a6-8fd0-1ea40cd73353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:23.320876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:23.330963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-61efe230-124c-4612-a6c8-4468372630ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e3fe38317c94c548147ee14f247c08a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.340837 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-61efe230-124c-4612-a6c8-4468372630ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e3fe38317c94c548147ee14f247c08a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.341619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-61efe230-124c-4612-a6c8-4468372630ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e3fe38317c94c548147ee14f247c08a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.342602 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:23.342985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-61efe230-124c-4612-a6c8-4468372630ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e3fe38317c94c548147ee14f247c08a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.365770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-61efe230-124c-4612-a6c8-4468372630ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e3fe38317c94c548147ee14f247c08a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.378059 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26cf674-94eb-43a6-8fd0-1ea40cd73353 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:23.387075 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 991/1992] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:23 2022] DELETE /identity/v3/users/5e3fe38317c94c548147ee14f247c08a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:23.398586 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504409e-1534-4ea4-a0bf-13b7ceb87608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:23.400040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504409e-1534-4ea4-a0bf-13b7ceb87608 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.400329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504409e-1534-4ea4-a0bf-13b7ceb87608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.400611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2504409e-1534-4ea4-a0bf-13b7ceb87608 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.401297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504409e-1534-4ea4-a0bf-13b7ceb87608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:23.403329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2504409e-1534-4ea4-a0bf-13b7ceb87608 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:23.436535 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1002/1993] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:23 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:23.437087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efe230-124c-4612-a6c8-4468372630ad admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:23.439437 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-61efe230-124c-4612-a6c8-4468372630ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:23.539564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61efe230-124c-4612-a6c8-4468372630ad admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:23.552728 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ffb60d-8ee2-4fc0-bcd2-30d6d47966f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:23.553395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffb60d-8ee2-4fc0-bcd2-30d6d47966f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.553679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffb60d-8ee2-4fc0-bcd2-30d6d47966f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.554016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffb60d-8ee2-4fc0-bcd2-30d6d47966f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.555370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ffb60d-8ee2-4fc0-bcd2-30d6d47966f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:23.557659 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ffb60d-8ee2-4fc0-bcd2-30d6d47966f9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:23.574461 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 992/1994] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:23 2022] GET /identity/v3/roles => generated 1427 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:23.576223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504409e-1534-4ea4-a0bf-13b7ceb87608 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:23.594820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2504409e-1534-4ea4-a0bf-13b7ceb87608 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:23.617132 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb849c3a-a179-4414-bbbf-742ad383514b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:23.617842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb849c3a-a179-4414-bbbf-742ad383514b admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.618649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb849c3a-a179-4414-bbbf-742ad383514b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.618970 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb849c3a-a179-4414-bbbf-742ad383514b admin admin] PATH_INFO: `/v3/projects/e8c4277e4c5e4ef28e8d48b0783dbed5/users/cbdca0842d8446bca7aa87d95eb4949f/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.640787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb849c3a-a179-4414-bbbf-742ad383514b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8c4277e4c5e4ef28e8d48b0783dbed5, user_id=cbdca0842d8446bca7aa87d95eb4949f, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:23.643699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb849c3a-a179-4414-bbbf-742ad383514b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:23.663377 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1003/1995] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:23 2022] PUT /identity/v3/projects/e8c4277e4c5e4ef28e8d48b0783dbed5/users/cbdca0842d8446bca7aa87d95eb4949f/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:23.667611 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ffb60d-8ee2-4fc0-bcd2-30d6d47966f9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:23.681937 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ffb60d-8ee2-4fc0-bcd2-30d6d47966f9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:23.764976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d5a21-24a4-4ab5-8c50-1e079c51e082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:23.765775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d5a21-24a4-4ab5-8c50-1e079c51e082 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.766062 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d5a21-24a4-4ab5-8c50-1e079c51e082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.766426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d5a21-24a4-4ab5-8c50-1e079c51e082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.767285 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d5a21-24a4-4ab5-8c50-1e079c51e082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:23.769406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d5a21-24a4-4ab5-8c50-1e079c51e082 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:23.777397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb849c3a-a179-4414-bbbf-742ad383514b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:23.779803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bb849c3a-a179-4414-bbbf-742ad383514b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:23.783036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 993/1996] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:23 2022] GET /identity/v3/roles => generated 1427 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:15:23.787377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d5a21-24a4-4ab5-8c50-1e079c51e082 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:23.796002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb849c3a-a179-4414-bbbf-742ad383514b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:23.801295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ed339-8384-49aa-8c2b-19bf8bfcf484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:23.801983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ed339-8384-49aa-8c2b-19bf8bfcf484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.802419 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ed339-8384-49aa-8c2b-19bf8bfcf484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.802750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ed339-8384-49aa-8c2b-19bf8bfcf484 admin admin] PATH_INFO: `/v3/projects/be3208c8758348a1af50872006a120cf` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.803858 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d5a21-24a4-4ab5-8c50-1e079c51e082 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:23.804164 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ed339-8384-49aa-8c2b-19bf8bfcf484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be3208c8758348a1af50872006a120cf)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:23.806897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ed339-8384-49aa-8c2b-19bf8bfcf484 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:23.810400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97547da4-0fb9-476b-9b6c-f43620c56243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:23.811232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97547da4-0fb9-476b-9b6c-f43620c56243 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.811644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97547da4-0fb9-476b-9b6c-f43620c56243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.811960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97547da4-0fb9-476b-9b6c-f43620c56243 admin admin] PATH_INFO: `/v3/projects/e8c4277e4c5e4ef28e8d48b0783dbed5/users/cbdca0842d8446bca7aa87d95eb4949f/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.814312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97547da4-0fb9-476b-9b6c-f43620c56243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8c4277e4c5e4ef28e8d48b0783dbed5, user_id=cbdca0842d8446bca7aa87d95eb4949f, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:23.817760 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97547da4-0fb9-476b-9b6c-f43620c56243 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:23.831457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 994/1997] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:23 2022] PUT /identity/v3/projects/e8c4277e4c5e4ef28e8d48b0783dbed5/users/cbdca0842d8446bca7aa87d95eb4949f/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:23.843442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4da5f8-d4cf-4b9d-8a42-4e8509cff330 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.843730 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4da5f8-d4cf-4b9d-8a42-4e8509cff330 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.844015 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4da5f8-d4cf-4b9d-8a42-4e8509cff330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.863729 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-138ed339-8384-49aa-8c2b-19bf8bfcf484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be3208c8758348a1af50872006a120cf', 'request_id': 'req-138ed339-8384-49aa-8c2b-19bf8bfcf484'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.871718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-138ed339-8384-49aa-8c2b-19bf8bfcf484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be3208c8758348a1af50872006a120cf', 'request_id': 'req-138ed339-8384-49aa-8c2b-19bf8bfcf484'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.875795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-6d4da5f8-d4cf-4b9d-8a42-4e8509cff330 None None] MFA Rules not processed for user `cbdca0842d8446bca7aa87d95eb4949f`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:23.878630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-138ed339-8384-49aa-8c2b-19bf8bfcf484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be3208c8758348a1af50872006a120cf', 'request_id': 'req-138ed339-8384-49aa-8c2b-19bf8bfcf484'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.878803 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6d4da5f8-d4cf-4b9d-8a42-4e8509cff330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:23.894512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-138ed339-8384-49aa-8c2b-19bf8bfcf484 admin admin] The token cache is being invalidate because project be3208c8758348a1af50872006a120cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:23.894973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-138ed339-8384-49aa-8c2b-19bf8bfcf484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.895642 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-138ed339-8384-49aa-8c2b-19bf8bfcf484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:23.901978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1004/1998] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:23 2022] DELETE /identity/v3/projects/be3208c8758348a1af50872006a120cf => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:23.903740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a0ed5-1ba9-4468-8575-300514886938 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.904028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a0ed5-1ba9-4468-8575-300514886938 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.904667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a0ed5-1ba9-4468-8575-300514886938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:23.910195 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:23.924439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f07a0ed5-1ba9-4468-8575-300514886938 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:23.925688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f07a0ed5-1ba9-4468-8575-300514886938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:23.957678 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 995/1999] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:23.971072 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4da5f8-d4cf-4b9d-8a42-4e8509cff330 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:23.972464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6d4da5f8-d4cf-4b9d-8a42-4e8509cff330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:23.990128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1005/2000] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:23.998442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d632bb1-ba59-4d87-8a24-f67f0a04567e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:23.998768 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d632bb1-ba59-4d87-8a24-f67f0a04567e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:23.999052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d632bb1-ba59-4d87-8a24-f67f0a04567e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.018353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-0d632bb1-ba59-4d87-8a24-f67f0a04567e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:24.019596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0d632bb1-ba59-4d87-8a24-f67f0a04567e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:24.027612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1006/2001] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:24.031243 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4da5f8-d4cf-4b9d-8a42-4e8509cff330 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.032035 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d632bb1-ba59-4d87-8a24-f67f0a04567e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:24.034926 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3f6d2-7609-4e15-910a-69d34c01d11c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.035555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3f6d2-7609-4e15-910a-69d34c01d11c service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.035838 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3f6d2-7609-4e15-910a-69d34c01d11c service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.036117 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3f6d2-7609-4e15-910a-69d34c01d11c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.041945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d632bb1-ba59-4d87-8a24-f67f0a04567e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.046381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3f6d2-7609-4e15-910a-69d34c01d11c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.046831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764d855-81d9-4e80-9682-168df8ed59b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.047441 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764d855-81d9-4e80-9682-168df8ed59b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.047718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764d855-81d9-4e80-9682-168df8ed59b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.047862 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3f6d2-7609-4e15-910a-69d34c01d11c service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.047990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764d855-81d9-4e80-9682-168df8ed59b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.048676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764d855-81d9-4e80-9682-168df8ed59b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:24.049098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764d855-81d9-4e80-9682-168df8ed59b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.050706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764d855-81d9-4e80-9682-168df8ed59b9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.058252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 996/2002] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:24.063527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1007/2003] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:15:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:15:24.068956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3f6d2-7609-4e15-910a-69d34c01d11c service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:24.079681 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3f6d2-7609-4e15-910a-69d34c01d11c service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.084620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83a624-cc73-4b18-83f4-da8b2a739e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.085239 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83a624-cc73-4b18-83f4-da8b2a739e14 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.085522 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83a624-cc73-4b18-83f4-da8b2a739e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.085798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a83a624-cc73-4b18-83f4-da8b2a739e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.086511 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83a624-cc73-4b18-83f4-da8b2a739e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.088491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a83a624-cc73-4b18-83f4-da8b2a739e14 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.118007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 997/2004] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:24 2022] POST /identity/v3/projects => generated 386 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:24.123042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764d855-81d9-4e80-9682-168df8ed59b9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:24.136896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764d855-81d9-4e80-9682-168df8ed59b9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.141870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:24.153640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83a624-cc73-4b18-83f4-da8b2a739e14 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:24.212030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c47d5-c333-4675-8c49-2c6a4ca3a16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.212819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c47d5-c333-4675-8c49-2c6a4ca3a16d admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.213215 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c47d5-c333-4675-8c49-2c6a4ca3a16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.213535 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c47d5-c333-4675-8c49-2c6a4ca3a16d admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.213909 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a83a624-cc73-4b18-83f4-da8b2a739e14 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.214314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c47d5-c333-4675-8c49-2c6a4ca3a16d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.216697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6c47d5-c333-4675-8c49-2c6a4ca3a16d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.217771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a8148-7d98-4ad1-a686-dbec290c8fdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.218398 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a8148-7d98-4ad1-a686-dbec290c8fdd service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.218710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a8148-7d98-4ad1-a686-dbec290c8fdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.218996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a8148-7d98-4ad1-a686-dbec290c8fdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.220953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d85a8148-7d98-4ad1-a686-dbec290c8fdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:24.240908 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1008/2005] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:24 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:24.246077 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c47d5-c333-4675-8c49-2c6a4ca3a16d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:24.255936 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c47d5-c333-4675-8c49-2c6a4ca3a16d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.260706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79756d43-a4d5-4b9a-9277-e3349078c402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.261325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79756d43-a4d5-4b9a-9277-e3349078c402 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.261606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79756d43-a4d5-4b9a-9277-e3349078c402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.261893 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79756d43-a4d5-4b9a-9277-e3349078c402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.262595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79756d43-a4d5-4b9a-9277-e3349078c402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.264223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79756d43-a4d5-4b9a-9277-e3349078c402 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.276889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1009/2006] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:24 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:15:24.280509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79756d43-a4d5-4b9a-9277-e3349078c402 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:24.288334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:24.291585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a8148-7d98-4ad1-a686-dbec290c8fdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.292226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79756d43-a4d5-4b9a-9277-e3349078c402 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.293279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a8148-7d98-4ad1-a686-dbec290c8fdd service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.297471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3fcba-364b-472b-95c0-865dc11e15a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.298325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3fcba-364b-472b-95c0-865dc11e15a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.298635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3fcba-364b-472b-95c0-865dc11e15a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.299497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3fcba-364b-472b-95c0-865dc11e15a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.299631 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3fcba-364b-472b-95c0-865dc11e15a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.301250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3fcba-364b-472b-95c0-865dc11e15a0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.306510 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 998/2007] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:24.313424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1010/2008] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:24 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:24.317782 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a8148-7d98-4ad1-a686-dbec290c8fdd service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:24.331744 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a8148-7d98-4ad1-a686-dbec290c8fdd service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.336663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cac821-f7f4-4555-b695-4e266b628947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.338190 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cac821-f7f4-4555-b695-4e266b628947 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.338775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cac821-f7f4-4555-b695-4e266b628947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.339138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cac821-f7f4-4555-b695-4e266b628947 admin admin] PATH_INFO: `/v3/projects/109da8094b524882b31f5a538d16280b/users/72b03dad5d7e4b4992dd10bd9b1b6b77/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.363141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cac821-f7f4-4555-b695-4e266b628947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=109da8094b524882b31f5a538d16280b, user_id=72b03dad5d7e4b4992dd10bd9b1b6b77, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.366123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cac821-f7f4-4555-b695-4e266b628947 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.383804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 999/2009] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:24 2022] PUT /identity/v3/projects/109da8094b524882b31f5a538d16280b/users/72b03dad5d7e4b4992dd10bd9b1b6b77/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:24.394567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4fa3d-003e-4578-be9c-56c7fdf71ea7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.394880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4fa3d-003e-4578-be9c-56c7fdf71ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.395151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e4fa3d-003e-4578-be9c-56c7fdf71ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.430805 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-86e4fa3d-003e-4578-be9c-56c7fdf71ea7 None None] MFA Rules not processed for user `72b03dad5d7e4b4992dd10bd9b1b6b77`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:24.432053 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-86e4fa3d-003e-4578-be9c-56c7fdf71ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:24.528132 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1011/2010] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:24.546858 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cac821-f7f4-4555-b695-4e266b628947 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:24.614582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cac821-f7f4-4555-b695-4e266b628947 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.618531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cfac1-80df-4aa0-8635-979f2971c0e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.619206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cfac1-80df-4aa0-8635-979f2971c0e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.619503 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cfac1-80df-4aa0-8635-979f2971c0e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.619770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cfac1-80df-4aa0-8635-979f2971c0e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.631385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5cfac1-80df-4aa0-8635-979f2971c0e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.633125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5cfac1-80df-4aa0-8635-979f2971c0e2 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.643786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1000/2011] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:24.646877 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4fa3d-003e-4578-be9c-56c7fdf71ea7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:24.648358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-86e4fa3d-003e-4578-be9c-56c7fdf71ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:24.666606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cfac1-80df-4aa0-8635-979f2971c0e2 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:24.754276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cfac1-80df-4aa0-8635-979f2971c0e2 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.760063 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e4fa3d-003e-4578-be9c-56c7fdf71ea7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:24.762780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d5b4e7-0e9a-4ed4-b371-99e850ca6f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.763469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b4e7-0e9a-4ed4-b371-99e850ca6f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.763741 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b4e7-0e9a-4ed4-b371-99e850ca6f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.764020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d5b4e7-0e9a-4ed4-b371-99e850ca6f90 admin admin] PATH_INFO: `/v3/users/72b03dad5d7e4b4992dd10bd9b1b6b77` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.765254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5b4e7-0e9a-4ed4-b371-99e850ca6f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72b03dad5d7e4b4992dd10bd9b1b6b77)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.767344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d5b4e7-0e9a-4ed4-b371-99e850ca6f90 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.767578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51651e-3d25-4ccb-a8a2-62f208676e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:24.768372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51651e-3d25-4ccb-a8a2-62f208676e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.768734 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51651e-3d25-4ccb-a8a2-62f208676e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.769151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51651e-3d25-4ccb-a8a2-62f208676e96 admin admin] PATH_INFO: `/v3/users/f5fb2a47a44942eba2237b15e1c1f4c9` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.770431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51651e-3d25-4ccb-a8a2-62f208676e96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5fb2a47a44942eba2237b15e1c1f4c9)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:24.772522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51651e-3d25-4ccb-a8a2-62f208676e96 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:24.779526 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:24.826751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-64d5b4e7-0e9a-4ed4-b371-99e850ca6f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72b03dad5d7e4b4992dd10bd9b1b6b77'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:24.838563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5e51651e-3d25-4ccb-a8a2-62f208676e96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5fb2a47a44942eba2237b15e1c1f4c9'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:24.839495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5e51651e-3d25-4ccb-a8a2-62f208676e96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5fb2a47a44942eba2237b15e1c1f4c9'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:24.840300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-64d5b4e7-0e9a-4ed4-b371-99e850ca6f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72b03dad5d7e4b4992dd10bd9b1b6b77'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:24.841228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-64d5b4e7-0e9a-4ed4-b371-99e850ca6f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72b03dad5d7e4b4992dd10bd9b1b6b77'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:24.842816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-64d5b4e7-0e9a-4ed4-b371-99e850ca6f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72b03dad5d7e4b4992dd10bd9b1b6b77'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:24.850807 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5e51651e-3d25-4ccb-a8a2-62f208676e96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5fb2a47a44942eba2237b15e1c1f4c9'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:24.851754 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5e51651e-3d25-4ccb-a8a2-62f208676e96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5fb2a47a44942eba2237b15e1c1f4c9'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:24.860245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-64d5b4e7-0e9a-4ed4-b371-99e850ca6f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72b03dad5d7e4b4992dd10bd9b1b6b77'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:24.866339 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:24.870436 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5e51651e-3d25-4ccb-a8a2-62f208676e96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5fb2a47a44942eba2237b15e1c1f4c9'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:24.887350 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1001/2012] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:24 2022] DELETE /identity/v3/users/72b03dad5d7e4b4992dd10bd9b1b6b77 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:24.888200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1012/2013] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:24 2022] DELETE /identity/v3/users/f5fb2a47a44942eba2237b15e1c1f4c9 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:24.892401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5602702-4e03-4e71-ac6d-5778e44377ec None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.892616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5602702-4e03-4e71-ac6d-5778e44377ec None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.892809 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f022fd-e547-45c8-ac45-1d04b1a76a6c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:24.892952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5602702-4e03-4e71-ac6d-5778e44377ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.893330 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f022fd-e547-45c8-ac45-1d04b1a76a6c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:24.893712 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f022fd-e547-45c8-ac45-1d04b1a76a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:24.917575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f5602702-4e03-4e71-ac6d-5778e44377ec None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:24.918861 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f5602702-4e03-4e71-ac6d-5778e44377ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:24.921622 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a6f022fd-e547-45c8-ac45-1d04b1a76a6c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:24.922946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a6f022fd-e547-45c8-ac45-1d04b1a76a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:24.996650 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1013/2014] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:25.002952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1002/2015] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:25.006831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f022fd-e547-45c8-ac45-1d04b1a76a6c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:25.008231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a6f022fd-e547-45c8-ac45-1d04b1a76a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:25.013349 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5602702-4e03-4e71-ac6d-5778e44377ec None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:25.014809 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f5602702-4e03-4e71-ac6d-5778e44377ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:25.068865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f022fd-e547-45c8-ac45-1d04b1a76a6c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:25.072561 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5602702-4e03-4e71-ac6d-5778e44377ec None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:25.073241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad299414-8863-45a6-b17c-5dfdc729ea6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:25.073883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad299414-8863-45a6-b17c-5dfdc729ea6b service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.074155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad299414-8863-45a6-b17c-5dfdc729ea6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.074436 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad299414-8863-45a6-b17c-5dfdc729ea6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.076486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cea5e-28e3-4af1-99a1-deeb8a178781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:25.077117 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cea5e-28e3-4af1-99a1-deeb8a178781 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.077400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cea5e-28e3-4af1-99a1-deeb8a178781 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.077675 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cea5e-28e3-4af1-99a1-deeb8a178781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.084507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad299414-8863-45a6-b17c-5dfdc729ea6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:25.085964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad299414-8863-45a6-b17c-5dfdc729ea6b service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:25.088107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cea5e-28e3-4af1-99a1-deeb8a178781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:25.089555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cea5e-28e3-4af1-99a1-deeb8a178781 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:25.096972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1014/2016] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:25.099808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1003/2017] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:25.402586 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad299414-8863-45a6-b17c-5dfdc729ea6b service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:25.404846 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ad299414-8863-45a6-b17c-5dfdc729ea6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:25.425084 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad299414-8863-45a6-b17c-5dfdc729ea6b service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:25.430130 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:25.431383 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.431698 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.431958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] PATH_INFO: `/v3/projects/bd7e1c1ac7c64876a9227bfff6aaf4c8` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.433394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd7e1c1ac7c64876a9227bfff6aaf4c8)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:25.435534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:25.463316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:25.494172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd7e1c1ac7c64876a9227bfff6aaf4c8', 'request_id': 'req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.501786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd7e1c1ac7c64876a9227bfff6aaf4c8', 'request_id': 'req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.509443 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd7e1c1ac7c64876a9227bfff6aaf4c8', 'request_id': 'req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.524177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] The token cache is being invalidate because project bd7e1c1ac7c64876a9227bfff6aaf4c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:25.525043 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.525827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.527445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1015/2018] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:25 2022] DELETE /identity/v3/projects/bd7e1c1ac7c64876a9227bfff6aaf4c8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:25.531549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cea5e-28e3-4af1-99a1-deeb8a178781 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:25.546314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cea5e-28e3-4af1-99a1-deeb8a178781 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:25.577877 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:25.579459 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:25.620587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc089c-064c-45f4-ae65-5bf6df99b35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:25.621317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc089c-064c-45f4-ae65-5bf6df99b35b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.621614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc089c-064c-45f4-ae65-5bf6df99b35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.621898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc089c-064c-45f4-ae65-5bf6df99b35b admin admin] PATH_INFO: `/v3/users/cbdca0842d8446bca7aa87d95eb4949f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.623316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc089c-064c-45f4-ae65-5bf6df99b35b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbdca0842d8446bca7aa87d95eb4949f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:25.625511 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc089c-064c-45f4-ae65-5bf6df99b35b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:25.669105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e1d7c-cb86-47e2-a0c4-023d7a9fae70 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:25.675092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2beb9-b491-45bd-942e-c491bfb2ecbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:25.676032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2beb9-b491-45bd-942e-c491bfb2ecbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.676410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2beb9-b491-45bd-942e-c491bfb2ecbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.676693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2beb9-b491-45bd-942e-c491bfb2ecbd admin admin] PATH_INFO: `/v3/projects/109da8094b524882b31f5a538d16280b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.677974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2beb9-b491-45bd-942e-c491bfb2ecbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=109da8094b524882b31f5a538d16280b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:25.680271 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2beb9-b491-45bd-942e-c491bfb2ecbd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:25.688087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:25.699837 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-49cc089c-064c-45f4-ae65-5bf6df99b35b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbdca0842d8446bca7aa87d95eb4949f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.709280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-49cc089c-064c-45f4-ae65-5bf6df99b35b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbdca0842d8446bca7aa87d95eb4949f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.710059 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-49cc089c-064c-45f4-ae65-5bf6df99b35b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbdca0842d8446bca7aa87d95eb4949f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.710780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-49cc089c-064c-45f4-ae65-5bf6df99b35b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbdca0842d8446bca7aa87d95eb4949f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.726476 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-49cc089c-064c-45f4-ae65-5bf6df99b35b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbdca0842d8446bca7aa87d95eb4949f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.743398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-00d2beb9-b491-45bd-942e-c491bfb2ecbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '109da8094b524882b31f5a538d16280b', 'request_id': 'req-00d2beb9-b491-45bd-942e-c491bfb2ecbd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.745152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1004/2019] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:25 2022] DELETE /identity/v3/users/cbdca0842d8446bca7aa87d95eb4949f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:25.750878 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-00d2beb9-b491-45bd-942e-c491bfb2ecbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '109da8094b524882b31f5a538d16280b', 'request_id': 'req-00d2beb9-b491-45bd-942e-c491bfb2ecbd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.758131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-00d2beb9-b491-45bd-942e-c491bfb2ecbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '109da8094b524882b31f5a538d16280b', 'request_id': 'req-00d2beb9-b491-45bd-942e-c491bfb2ecbd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.784631 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-00d2beb9-b491-45bd-942e-c491bfb2ecbd admin admin] The token cache is being invalidate because project 109da8094b524882b31f5a538d16280b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:25.784927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-00d2beb9-b491-45bd-942e-c491bfb2ecbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.786172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-00d2beb9-b491-45bd-942e-c491bfb2ecbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:25.788335 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1016/2020] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:25 2022] DELETE /identity/v3/projects/109da8094b524882b31f5a538d16280b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:25.798250 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa8bbc-9bab-4428-b06e-9a7278c388ae None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.798645 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa8bbc-9bab-4428-b06e-9a7278c388ae None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.799029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa8bbc-9bab-4428-b06e-9a7278c388ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.822271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c8aa8bbc-9bab-4428-b06e-9a7278c388ae None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:25.823643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c8aa8bbc-9bab-4428-b06e-9a7278c388ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:25.924569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1005/2021] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:25.932920 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57290eaa-c43f-47de-93c7-5d63643f61e0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:25.933357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57290eaa-c43f-47de-93c7-5d63643f61e0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:25.933662 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57290eaa-c43f-47de-93c7-5d63643f61e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:25.991902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-57290eaa-c43f-47de-93c7-5d63643f61e0 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:25.994332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-57290eaa-c43f-47de-93c7-5d63643f61e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.010769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1017/2022] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:26.017047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa8bbc-9bab-4428-b06e-9a7278c388ae None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.028925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa8bbc-9bab-4428-b06e-9a7278c388ae None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.044992 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964b0d7-47dc-4348-98ff-9735912a5e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.046032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964b0d7-47dc-4348-98ff-9735912a5e6c admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.046645 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964b0d7-47dc-4348-98ff-9735912a5e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.047354 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964b0d7-47dc-4348-98ff-9735912a5e6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.048729 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964b0d7-47dc-4348-98ff-9735912a5e6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:26.049036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964b0d7-47dc-4348-98ff-9735912a5e6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.051312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964b0d7-47dc-4348-98ff-9735912a5e6c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.073333 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1006/2023] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:15:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:15:26.080020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57290eaa-c43f-47de-93c7-5d63643f61e0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.094953 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57290eaa-c43f-47de-93c7-5d63643f61e0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.106845 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1ffef-ef37-41a8-9a6e-d5573684b884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.108297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1ffef-ef37-41a8-9a6e-d5573684b884 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.109438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1ffef-ef37-41a8-9a6e-d5573684b884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.110252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a1ffef-ef37-41a8-9a6e-d5573684b884 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.111316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1ffef-ef37-41a8-9a6e-d5573684b884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.114652 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a1ffef-ef37-41a8-9a6e-d5573684b884 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.158026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1018/2024] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:26 2022] POST /identity/v3/projects => generated 378 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:26.159184 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964b0d7-47dc-4348-98ff-9735912a5e6c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.159463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1ffef-ef37-41a8-9a6e-d5573684b884 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.162129 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-46a1ffef-ef37-41a8-9a6e-d5573684b884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.172076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964b0d7-47dc-4348-98ff-9735912a5e6c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.185527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:26.254344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a1ffef-ef37-41a8-9a6e-d5573684b884 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.256117 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690df98-849c-45cf-b48b-e077f284c0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.256754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690df98-849c-45cf-b48b-e077f284c0a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.257039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690df98-849c-45cf-b48b-e077f284c0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.257317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690df98-849c-45cf-b48b-e077f284c0a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.257999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690df98-849c-45cf-b48b-e077f284c0a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.259197 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121e243-a8ec-42cd-a24d-fe48e04b2f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.259854 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121e243-a8ec-42cd-a24d-fe48e04b2f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.260018 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690df98-849c-45cf-b48b-e077f284c0a2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.260146 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121e243-a8ec-42cd-a24d-fe48e04b2f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.260426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121e243-a8ec-42cd-a24d-fe48e04b2f70 admin admin] PATH_INFO: `/v3/projects/e8c4277e4c5e4ef28e8d48b0783dbed5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.261617 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121e243-a8ec-42cd-a24d-fe48e04b2f70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8c4277e4c5e4ef28e8d48b0783dbed5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.263600 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121e243-a8ec-42cd-a24d-fe48e04b2f70 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.282909 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1007/2025] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:26 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:26.288004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690df98-849c-45cf-b48b-e077f284c0a2 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.301859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690df98-849c-45cf-b48b-e077f284c0a2 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.308088 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a2263-4402-40a7-97b7-e046267047ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.308225 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a2263-4402-40a7-97b7-e046267047ce admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.308534 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a2263-4402-40a7-97b7-e046267047ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.308847 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a2263-4402-40a7-97b7-e046267047ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.309652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a2263-4402-40a7-97b7-e046267047ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.311230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a2263-4402-40a7-97b7-e046267047ce admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.320549 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5121e243-a8ec-42cd-a24d-fe48e04b2f70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8c4277e4c5e4ef28e8d48b0783dbed5', 'request_id': 'req-5121e243-a8ec-42cd-a24d-fe48e04b2f70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:26.323896 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1008/2026] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:26 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:26.327715 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a2263-4402-40a7-97b7-e046267047ce admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.327870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5121e243-a8ec-42cd-a24d-fe48e04b2f70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8c4277e4c5e4ef28e8d48b0783dbed5', 'request_id': 'req-5121e243-a8ec-42cd-a24d-fe48e04b2f70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:26.333704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5121e243-a8ec-42cd-a24d-fe48e04b2f70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8c4277e4c5e4ef28e8d48b0783dbed5', 'request_id': 'req-5121e243-a8ec-42cd-a24d-fe48e04b2f70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:26.338916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a2263-4402-40a7-97b7-e046267047ce admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.348241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5121e243-a8ec-42cd-a24d-fe48e04b2f70 admin admin] The token cache is being invalidate because project e8c4277e4c5e4ef28e8d48b0783dbed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:26.348543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5121e243-a8ec-42cd-a24d-fe48e04b2f70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:26.349193 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5121e243-a8ec-42cd-a24d-fe48e04b2f70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:26.350777 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1019/2027] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:26 2022] DELETE /identity/v3/projects/e8c4277e4c5e4ef28e8d48b0783dbed5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:26.356781 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a886b2f-c550-45b0-a158-663710c730a6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.357076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a886b2f-c550-45b0-a158-663710c730a6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.357366 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a886b2f-c550-45b0-a158-663710c730a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.359175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:26.378634 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3a886b2f-c550-45b0-a158-663710c730a6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:26.379895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3a886b2f-c550-45b0-a158-663710c730a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.401596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d7e62-cac8-424d-8bc9-0788f91a4499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.402221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d7e62-cac8-424d-8bc9-0788f91a4499 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.402507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d7e62-cac8-424d-8bc9-0788f91a4499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.402825 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d7e62-cac8-424d-8bc9-0788f91a4499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.403519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d7e62-cac8-424d-8bc9-0788f91a4499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.405128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d7e62-cac8-424d-8bc9-0788f91a4499 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.415884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1009/2028] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:26 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:15:26.420071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d7e62-cac8-424d-8bc9-0788f91a4499 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.422000 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-346d7e62-cac8-424d-8bc9-0788f91a4499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.435827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d7e62-cac8-424d-8bc9-0788f91a4499 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.441029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a80a5-0bac-4ff8-8c69-14ce3c786d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.441687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a80a5-0bac-4ff8-8c69-14ce3c786d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.441964 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a80a5-0bac-4ff8-8c69-14ce3c786d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.442479 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a80a5-0bac-4ff8-8c69-14ce3c786d77 admin admin] PATH_INFO: `/v3/projects/3117c1d2e4094902a026144869570e13/users/12c94145521d4662b87a478750905362/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.442590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1020/2029] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:26.450476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0221b-7f1a-43f2-86f3-4c8f3fd8c0cc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.450812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0221b-7f1a-43f2-86f3-4c8f3fd8c0cc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.451092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0221b-7f1a-43f2-86f3-4c8f3fd8c0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.458148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a80a5-0bac-4ff8-8c69-14ce3c786d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3117c1d2e4094902a026144869570e13, user_id=12c94145521d4662b87a478750905362, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.460851 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a80a5-0bac-4ff8-8c69-14ce3c786d77 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.471457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c5e0221b-7f1a-43f2-86f3-4c8f3fd8c0cc None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:26.472670 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c5e0221b-7f1a-43f2-86f3-4c8f3fd8c0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.475780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1010/2030] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:26 2022] PUT /identity/v3/projects/3117c1d2e4094902a026144869570e13/users/12c94145521d4662b87a478750905362/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:26.486394 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f3dc4-3400-460e-be92-f14d39b3772f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.486600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f3dc4-3400-460e-be92-f14d39b3772f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.486914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f3dc4-3400-460e-be92-f14d39b3772f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.513979 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:26.514708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-505f3dc4-3400-460e-be92-f14d39b3772f None None] MFA Rules not processed for user `12c94145521d4662b87a478750905362`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:26.515956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-505f3dc4-3400-460e-be92-f14d39b3772f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.547215 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1021/2031] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:26.551486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0221b-7f1a-43f2-86f3-4c8f3fd8c0cc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.561199 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0221b-7f1a-43f2-86f3-4c8f3fd8c0cc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.566059 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07cf64-35ad-4551-ba66-92f39a868c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.566746 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07cf64-35ad-4551-ba66-92f39a868c54 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.567026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07cf64-35ad-4551-ba66-92f39a868c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.567304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07cf64-35ad-4551-ba66-92f39a868c54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.568001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07cf64-35ad-4551-ba66-92f39a868c54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:26.568307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07cf64-35ad-4551-ba66-92f39a868c54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.569911 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07cf64-35ad-4551-ba66-92f39a868c54 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.585261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1022/2032] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:15:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:15:26.585436 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1011/2033] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:26.587600 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07cf64-35ad-4551-ba66-92f39a868c54 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.589768 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f3dc4-3400-460e-be92-f14d39b3772f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.598101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07cf64-35ad-4551-ba66-92f39a868c54 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.600745 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505f3dc4-3400-460e-be92-f14d39b3772f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.605106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270ba20-2194-4b60-b030-2dc793654e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.605720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270ba20-2194-4b60-b030-2dc793654e8b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.606115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270ba20-2194-4b60-b030-2dc793654e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.606299 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c97b32-0573-46ed-9779-afb4f47637af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.606410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270ba20-2194-4b60-b030-2dc793654e8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.607074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270ba20-2194-4b60-b030-2dc793654e8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.607862 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c97b32-0573-46ed-9779-afb4f47637af admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.608167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c97b32-0573-46ed-9779-afb4f47637af admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.608452 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c97b32-0573-46ed-9779-afb4f47637af admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.608974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270ba20-2194-4b60-b030-2dc793654e8b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.609474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c97b32-0573-46ed-9779-afb4f47637af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.611422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c97b32-0573-46ed-9779-afb4f47637af admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.641443 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1023/2034] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:26 2022] POST /identity/v3/projects => generated 378 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:26.642037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1012/2035] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:26 2022] POST /identity/v3/projects => generated 372 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:26.642342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270ba20-2194-4b60-b030-2dc793654e8b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.645734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c97b32-0573-46ed-9779-afb4f47637af admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.707883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270ba20-2194-4b60-b030-2dc793654e8b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.710396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c97b32-0573-46ed-9779-afb4f47637af admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.713154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbe1fe-e5ab-4ac4-9fa4-b1fb9af582cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.713783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbe1fe-e5ab-4ac4-9fa4-b1fb9af582cc admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.714062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbe1fe-e5ab-4ac4-9fa4-b1fb9af582cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.714344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbe1fe-e5ab-4ac4-9fa4-b1fb9af582cc admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.715046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbe1fe-e5ab-4ac4-9fa4-b1fb9af582cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.715954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06150e5-3cec-4618-9668-0079dbe34551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.716699 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06150e5-3cec-4618-9668-0079dbe34551 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.716981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06150e5-3cec-4618-9668-0079dbe34551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.717096 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbe1fe-e5ab-4ac4-9fa4-b1fb9af582cc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.717255 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06150e5-3cec-4618-9668-0079dbe34551 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.717919 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06150e5-3cec-4618-9668-0079dbe34551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.719883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06150e5-3cec-4618-9668-0079dbe34551 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.730374 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:26.739741 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1024/2036] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:26 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:26.746076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1013/2037] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:26 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:26.746218 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbe1fe-e5ab-4ac4-9fa4-b1fb9af582cc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.747496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06150e5-3cec-4618-9668-0079dbe34551 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.753626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbe1fe-e5ab-4ac4-9fa4-b1fb9af582cc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.757296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06150e5-3cec-4618-9668-0079dbe34551 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.758278 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc703a-c1c3-4c3a-83ed-8ebd1e3f5e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.758937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc703a-c1c3-4c3a-83ed-8ebd1e3f5e05 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.759219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc703a-c1c3-4c3a-83ed-8ebd1e3f5e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.759492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc703a-c1c3-4c3a-83ed-8ebd1e3f5e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.760178 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc703a-c1c3-4c3a-83ed-8ebd1e3f5e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.761757 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc703a-c1c3-4c3a-83ed-8ebd1e3f5e05 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.762114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10758b-78a4-4e64-856a-79b3c852893e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.762803 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10758b-78a4-4e64-856a-79b3c852893e admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.763089 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10758b-78a4-4e64-856a-79b3c852893e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.763368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10758b-78a4-4e64-856a-79b3c852893e admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.764060 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10758b-78a4-4e64-856a-79b3c852893e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.765659 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10758b-78a4-4e64-856a-79b3c852893e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.773247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1025/2038] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:26 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:15:26.776437 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1014/2039] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:26 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:15:26.777788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc703a-c1c3-4c3a-83ed-8ebd1e3f5e05 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.780192 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10758b-78a4-4e64-856a-79b3c852893e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.787991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc703a-c1c3-4c3a-83ed-8ebd1e3f5e05 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.791694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10758b-78a4-4e64-856a-79b3c852893e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.793232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04435f87-9587-4aab-b80e-3dbdfa16acd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.793836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04435f87-9587-4aab-b80e-3dbdfa16acd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.794449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04435f87-9587-4aab-b80e-3dbdfa16acd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.794761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04435f87-9587-4aab-b80e-3dbdfa16acd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.795465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04435f87-9587-4aab-b80e-3dbdfa16acd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.796788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cbb0a-32c6-48f7-bf22-ea7f0b87de14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.797069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04435f87-9587-4aab-b80e-3dbdfa16acd6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.798770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cbb0a-32c6-48f7-bf22-ea7f0b87de14 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.798770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cbb0a-32c6-48f7-bf22-ea7f0b87de14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.798770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cbb0a-32c6-48f7-bf22-ea7f0b87de14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.798999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cbb0a-32c6-48f7-bf22-ea7f0b87de14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.801145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1cbb0a-32c6-48f7-bf22-ea7f0b87de14 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.811360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1026/2040] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:26 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:15:26.813766 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1015/2041] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:26 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:15:26.815511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04435f87-9587-4aab-b80e-3dbdfa16acd6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.817612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cbb0a-32c6-48f7-bf22-ea7f0b87de14 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:26.827409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1cbb0a-32c6-48f7-bf22-ea7f0b87de14 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.828261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04435f87-9587-4aab-b80e-3dbdfa16acd6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:26.832267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac818326-5426-4a4b-a7fc-6612c9e6fca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.832920 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac818326-5426-4a4b-a7fc-6612c9e6fca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.833199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac818326-5426-4a4b-a7fc-6612c9e6fca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.833303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1091e-92ee-414f-8d0f-ddc4679d3216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:26.833469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac818326-5426-4a4b-a7fc-6612c9e6fca5 admin admin] PATH_INFO: `/v3/projects/5cbef8ed086d4e8c821483cdf2b4c607/users/245c0de3d5de42e3b33e261ab635fe95/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.833864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1091e-92ee-414f-8d0f-ddc4679d3216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.834153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1091e-92ee-414f-8d0f-ddc4679d3216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.834432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1091e-92ee-414f-8d0f-ddc4679d3216 admin admin] PATH_INFO: `/v3/projects/354d6793216e4f87813f786a08a9188b/users/855168ad1d974de48904e33531b5a219/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.848845 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac818326-5426-4a4b-a7fc-6612c9e6fca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbef8ed086d4e8c821483cdf2b4c607, user_id=245c0de3d5de42e3b33e261ab635fe95, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.849808 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1091e-92ee-414f-8d0f-ddc4679d3216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=354d6793216e4f87813f786a08a9188b, user_id=855168ad1d974de48904e33531b5a219, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:26.851534 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac818326-5426-4a4b-a7fc-6612c9e6fca5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.853271 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1091e-92ee-414f-8d0f-ddc4679d3216 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:26.866717 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1027/2042] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:26 2022] PUT /identity/v3/projects/354d6793216e4f87813f786a08a9188b/users/855168ad1d974de48904e33531b5a219/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:15:26.867054 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1016/2043] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:26 2022] PUT /identity/v3/projects/5cbef8ed086d4e8c821483cdf2b4c607/users/245c0de3d5de42e3b33e261ab635fe95/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:26.878115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553a2ef-db7e-45f4-a711-b9c2dd5a2745 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.878281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7065d5-1270-49ca-9131-9a87e7a45e8e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:26.878406 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553a2ef-db7e-45f4-a711-b9c2dd5a2745 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.878507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7065d5-1270-49ca-9131-9a87e7a45e8e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:26.878708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553a2ef-db7e-45f4-a711-b9c2dd5a2745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.878853 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7065d5-1270-49ca-9131-9a87e7a45e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:26.908144 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-8553a2ef-db7e-45f4-a711-b9c2dd5a2745 None None] MFA Rules not processed for user `245c0de3d5de42e3b33e261ab635fe95`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:26.909377 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8553a2ef-db7e-45f4-a711-b9c2dd5a2745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.911337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9e7065d5-1270-49ca-9131-9a87e7a45e8e None None] MFA Rules not processed for user `855168ad1d974de48904e33531b5a219`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:26.912691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9e7065d5-1270-49ca-9131-9a87e7a45e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:26.953078 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:26.988232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1017/2044] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:26.995464 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1028/2045] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:26.999777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7065d5-1270-49ca-9131-9a87e7a45e8e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.012441 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553a2ef-db7e-45f4-a711-b9c2dd5a2745 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.017936 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8553a2ef-db7e-45f4-a711-b9c2dd5a2745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:27.110524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7065d5-1270-49ca-9131-9a87e7a45e8e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.116661 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f28d7-237b-4fd1-b8da-f58a01b412fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.117438 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f28d7-237b-4fd1-b8da-f58a01b412fd admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.117729 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f28d7-237b-4fd1-b8da-f58a01b412fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.118048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1f28d7-237b-4fd1-b8da-f58a01b412fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.118955 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f28d7-237b-4fd1-b8da-f58a01b412fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.121911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1f28d7-237b-4fd1-b8da-f58a01b412fd admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.143218 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553a2ef-db7e-45f4-a711-b9c2dd5a2745 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.151168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f29577-d5ac-4d1a-b1c4-05c83d73680c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e75edfc5bb404d63b27a355d5ab8af01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.152207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f29577-d5ac-4d1a-b1c4-05c83d73680c service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.152627 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f29577-d5ac-4d1a-b1c4-05c83d73680c service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.153179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f29577-d5ac-4d1a-b1c4-05c83d73680c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.159197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1018/2046] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:26 2022] POST /identity/v3/projects => generated 378 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:27.163219 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f28d7-237b-4fd1-b8da-f58a01b412fd admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.167168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f29577-d5ac-4d1a-b1c4-05c83d73680c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.168909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f29577-d5ac-4d1a-b1c4-05c83d73680c service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.183182 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1029/2047] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:27.203101 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:27.275001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1f28d7-237b-4fd1-b8da-f58a01b412fd admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.280525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57948e67-7c67-445e-953b-7c1cd9f27c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.282446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57948e67-7c67-445e-953b-7c1cd9f27c83 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.282786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57948e67-7c67-445e-953b-7c1cd9f27c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.283123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57948e67-7c67-445e-953b-7c1cd9f27c83 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.283872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57948e67-7c67-445e-953b-7c1cd9f27c83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.286529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57948e67-7c67-445e-953b-7c1cd9f27c83 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.314717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1019/2048] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:27 2022] POST /identity/v3/users => generated 296 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:27.318868 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f29577-d5ac-4d1a-b1c4-05c83d73680c service glance] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.330699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f29577-d5ac-4d1a-b1c4-05c83d73680c service glance] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.336109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998c792-987f-42c5-8b96-f1f5900b3320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.337279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998c792-987f-42c5-8b96-f1f5900b3320 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.337556 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998c792-987f-42c5-8b96-f1f5900b3320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.337832 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998c792-987f-42c5-8b96-f1f5900b3320 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.338529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998c792-987f-42c5-8b96-f1f5900b3320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.340182 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998c792-987f-42c5-8b96-f1f5900b3320 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.352263 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1030/2049] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:27 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:27.357098 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57948e67-7c67-445e-953b-7c1cd9f27c83 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.374864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57948e67-7c67-445e-953b-7c1cd9f27c83 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.380555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963c5b1-c6b9-41ba-8aeb-d59055379f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.381115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963c5b1-c6b9-41ba-8aeb-d59055379f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.381508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963c5b1-c6b9-41ba-8aeb-d59055379f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.381947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963c5b1-c6b9-41ba-8aeb-d59055379f0b admin admin] PATH_INFO: `/v3/projects/5618e2b6aacd4134b9ae78079ef614e4/users/b0e27636ad904f378e3dc98be0b1d957/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.398452 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963c5b1-c6b9-41ba-8aeb-d59055379f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5618e2b6aacd4134b9ae78079ef614e4, user_id=b0e27636ad904f378e3dc98be0b1d957, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.401788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963c5b1-c6b9-41ba-8aeb-d59055379f0b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.417059 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1020/2050] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:27 2022] PUT /identity/v3/projects/5618e2b6aacd4134b9ae78079ef614e4/users/b0e27636ad904f378e3dc98be0b1d957/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:27.420880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998c792-987f-42c5-8b96-f1f5900b3320 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.431869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998c792-987f-42c5-8b96-f1f5900b3320 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.503394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae9109-2b1c-42e8-a7e1-79ee823f8e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.504296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae9109-2b1c-42e8-a7e1-79ee823f8e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.504585 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae9109-2b1c-42e8-a7e1-79ee823f8e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.504869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae9109-2b1c-42e8-a7e1-79ee823f8e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.505563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae9109-2b1c-42e8-a7e1-79ee823f8e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.507205 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae9109-2b1c-42e8-a7e1-79ee823f8e34 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.517696 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1031/2051] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:27 2022] GET /identity/v3/roles => generated 1427 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:27.521876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963c5b1-c6b9-41ba-8aeb-d59055379f0b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.534706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963c5b1-c6b9-41ba-8aeb-d59055379f0b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.553519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d1e17-fb56-4633-8a09-e63218748119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.553853 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d1e17-fb56-4633-8a09-e63218748119 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.554142 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d1e17-fb56-4633-8a09-e63218748119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.554576 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d1e17-fb56-4633-8a09-e63218748119 admin admin] PATH_INFO: `/v3/projects/5618e2b6aacd4134b9ae78079ef614e4/users/b0e27636ad904f378e3dc98be0b1d957/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.557074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d1e17-fb56-4633-8a09-e63218748119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5618e2b6aacd4134b9ae78079ef614e4, user_id=b0e27636ad904f378e3dc98be0b1d957, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.560242 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d1e17-fb56-4633-8a09-e63218748119 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.573142 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1021/2052] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:27 2022] PUT /identity/v3/projects/5618e2b6aacd4134b9ae78079ef614e4/users/b0e27636ad904f378e3dc98be0b1d957/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:27.584268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc644d-75e5-447d-b85b-2bcd2b079ebd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.584560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc644d-75e5-447d-b85b-2bcd2b079ebd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.584920 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc644d-75e5-447d-b85b-2bcd2b079ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.614070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2bdc644d-75e5-447d-b85b-2bcd2b079ebd None None] MFA Rules not processed for user `b0e27636ad904f378e3dc98be0b1d957`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:27.615336 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2bdc644d-75e5-447d-b85b-2bcd2b079ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:27.662559 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:27.696838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1032/2053] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:27.709519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d1e17-fb56-4633-8a09-e63218748119 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:27.715897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-219d1e17-fb56-4633-8a09-e63218748119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:27.779680 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d1e17-fb56-4633-8a09-e63218748119 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:27.783424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ff0a5-64a1-46b3-bba7-3c9eb4b39c8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:27.784039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ff0a5-64a1-46b3-bba7-3c9eb4b39c8e service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:27.784316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ff0a5-64a1-46b3-bba7-3c9eb4b39c8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:27.784587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ff0a5-64a1-46b3-bba7-3c9eb4b39c8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:27.843107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ff0a5-64a1-46b3-bba7-3c9eb4b39c8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:27.844540 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ff0a5-64a1-46b3-bba7-3c9eb4b39c8e service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:27.853507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1022/2054] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:29.216259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73f42c-17f3-452c-848f-d880f49d9e7f None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:29.216259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73f42c-17f3-452c-848f-d880f49d9e7f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:29.216259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73f42c-17f3-452c-848f-d880f49d9e7f None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:29.216259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1033/2055] 10.222.0.100 () {54 vars in 926 bytes} [Mon Aug 1 15:15:29 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:15:29.225262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa5616-19d3-4975-857a-b537cce41759 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:29.226399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa5616-19d3-4975-857a-b537cce41759 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:29.226897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa5616-19d3-4975-857a-b537cce41759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:29.250883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2ffa5616-19d3-4975-857a-b537cce41759 None None] MFA Rules not processed for user `ec50ac7c54324884a1b2a6c63d617907`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:29.252528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2ffa5616-19d3-4975-857a-b537cce41759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:29.281979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:29.349985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1023/2056] 10.222.0.100 () {58 vars in 1023 bytes} [Mon Aug 1 15:15:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:15:29.362215 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73f42c-17f3-452c-848f-d880f49d9e7f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:29.363895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6d73f42c-17f3-452c-848f-d880f49d9e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:29.381296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73f42c-17f3-452c-848f-d880f49d9e7f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:29.387334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acd3e2-58e8-4acd-8e27-ab7dd039df93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:29.387948 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acd3e2-58e8-4acd-8e27-ab7dd039df93 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:29.388230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acd3e2-58e8-4acd-8e27-ab7dd039df93 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:29.388509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45acd3e2-58e8-4acd-8e27-ab7dd039df93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:29.397488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acd3e2-58e8-4acd-8e27-ab7dd039df93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:29.399733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45acd3e2-58e8-4acd-8e27-ab7dd039df93 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:29.410550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1034/2057] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:15:39.340233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=19 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:39.406345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=18 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:43.091986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa5616-19d3-4975-857a-b537cce41759 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:43.093419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2ffa5616-19d3-4975-857a-b537cce41759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:43.104984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa5616-19d3-4975-857a-b537cce41759 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:43.108708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45a69d-9143-49d0-adea-b6413bf8d6ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:43.109379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a69d-9143-49d0-adea-b6413bf8d6ec service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:43.109658 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a69d-9143-49d0-adea-b6413bf8d6ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:43.109931 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd45a69d-9143-49d0-adea-b6413bf8d6ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:43.169254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45a69d-9143-49d0-adea-b6413bf8d6ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:43.170740 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd45a69d-9143-49d0-adea-b6413bf8d6ec service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:43.180859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1024/2058] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:43.865163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acd3e2-58e8-4acd-8e27-ab7dd039df93 service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:43.875216 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45acd3e2-58e8-4acd-8e27-ab7dd039df93 service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:43.879168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59a04b-17b5-4912-87d8-27fa59263cc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:43.879786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59a04b-17b5-4912-87d8-27fa59263cc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:43.880067 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59a04b-17b5-4912-87d8-27fa59263cc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:43.880345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59a04b-17b5-4912-87d8-27fa59263cc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:43.891144 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59a04b-17b5-4912-87d8-27fa59263cc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:43.892614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59a04b-17b5-4912-87d8-27fa59263cc5 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:43.902724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1035/2059] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:53.175888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=28 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:53.898062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:55.729417 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45a69d-9143-49d0-adea-b6413bf8d6ec service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:55.799914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd45a69d-9143-49d0-adea-b6413bf8d6ec service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:55.804645 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077d3f3-c5e6-46e3-b250-5f4f837b50e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:55.805321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077d3f3-c5e6-46e3-b250-5f4f837b50e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:55.805734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077d3f3-c5e6-46e3-b250-5f4f837b50e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:55.806016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077d3f3-c5e6-46e3-b250-5f4f837b50e0 admin admin] PATH_INFO: `/v3/users/12c94145521d4662b87a478750905362` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:55.807240 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077d3f3-c5e6-46e3-b250-5f4f837b50e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12c94145521d4662b87a478750905362)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:55.809395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077d3f3-c5e6-46e3-b250-5f4f837b50e0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:55.880768 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b077d3f3-c5e6-46e3-b250-5f4f837b50e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12c94145521d4662b87a478750905362'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:55.891715 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b077d3f3-c5e6-46e3-b250-5f4f837b50e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12c94145521d4662b87a478750905362'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:55.892463 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b077d3f3-c5e6-46e3-b250-5f4f837b50e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12c94145521d4662b87a478750905362'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:55.893169 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b077d3f3-c5e6-46e3-b250-5f4f837b50e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12c94145521d4662b87a478750905362'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:55.917871 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b077d3f3-c5e6-46e3-b250-5f4f837b50e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12c94145521d4662b87a478750905362'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:55.934271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1025/2060] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:55 2022] DELETE /identity/v3/users/12c94145521d4662b87a478750905362 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:55.938614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e594743-c1d9-4c3b-95e4-6cca3bd70d4e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:55.939161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e594743-c1d9-4c3b-95e4-6cca3bd70d4e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:55.939246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e594743-c1d9-4c3b-95e4-6cca3bd70d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:55.959274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1e594743-c1d9-4c3b-95e4-6cca3bd70d4e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:55.961092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1e594743-c1d9-4c3b-95e4-6cca3bd70d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:56.072860 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1036/2061] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:56.113032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077d3f3-c5e6-46e3-b250-5f4f837b50e0 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:56.114622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b077d3f3-c5e6-46e3-b250-5f4f837b50e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:56.181959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077d3f3-c5e6-46e3-b250-5f4f837b50e0 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:56.186797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff8f7a-f0b2-4d67-a135-e142c2c484ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:56.187520 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff8f7a-f0b2-4d67-a135-e142c2c484ec service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:56.187883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff8f7a-f0b2-4d67-a135-e142c2c484ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:56.188241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff8f7a-f0b2-4d67-a135-e142c2c484ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:56.205033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff8f7a-f0b2-4d67-a135-e142c2c484ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:56.206890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff8f7a-f0b2-4d67-a135-e142c2c484ec service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:56.224463 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1026/2062] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:56.688921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e594743-c1d9-4c3b-95e4-6cca3bd70d4e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:56.690447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1e594743-c1d9-4c3b-95e4-6cca3bd70d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:56.713212 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e594743-c1d9-4c3b-95e4-6cca3bd70d4e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:56.719654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:56.720935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:56.721530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:56.722320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] PATH_INFO: `/v3/projects/3117c1d2e4094902a026144869570e13` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:56.724948 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3117c1d2e4094902a026144869570e13)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:56.727307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:56.778544 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3117c1d2e4094902a026144869570e13', 'request_id': 'req-bfa134b5-598d-44c5-a108-13d172fe4beb'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:56.785664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3117c1d2e4094902a026144869570e13', 'request_id': 'req-bfa134b5-598d-44c5-a108-13d172fe4beb'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:56.791626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3117c1d2e4094902a026144869570e13', 'request_id': 'req-bfa134b5-598d-44c5-a108-13d172fe4beb'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:56.805666 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] The token cache is being invalidate because project 3117c1d2e4094902a026144869570e13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:56.805896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:56.806546 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:56.808096 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1037/2063] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:56 2022] DELETE /identity/v3/projects/3117c1d2e4094902a026144869570e13 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:56.812034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff8f7a-f0b2-4d67-a135-e142c2c484ec service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:56.822396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff8f7a-f0b2-4d67-a135-e142c2c484ec service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:56.843751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:56.888449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d81b9-cad2-4aed-8565-b6f746b370ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:56.889097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d81b9-cad2-4aed-8565-b6f746b370ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:56.889425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d81b9-cad2-4aed-8565-b6f746b370ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:56.889670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d81b9-cad2-4aed-8565-b6f746b370ee admin admin] PATH_INFO: `/v3/users/245c0de3d5de42e3b33e261ab635fe95` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:56.891220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d81b9-cad2-4aed-8565-b6f746b370ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=245c0de3d5de42e3b33e261ab635fe95)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:56.893294 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d81b9-cad2-4aed-8565-b6f746b370ee admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:56.950893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a63d81b9-cad2-4aed-8565-b6f746b370ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '245c0de3d5de42e3b33e261ab635fe95'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:56.959392 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a63d81b9-cad2-4aed-8565-b6f746b370ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '245c0de3d5de42e3b33e261ab635fe95'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:56.959996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a63d81b9-cad2-4aed-8565-b6f746b370ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '245c0de3d5de42e3b33e261ab635fe95'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:56.961242 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a63d81b9-cad2-4aed-8565-b6f746b370ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '245c0de3d5de42e3b33e261ab635fe95'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:56.975360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a63d81b9-cad2-4aed-8565-b6f746b370ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '245c0de3d5de42e3b33e261ab635fe95'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:56.987823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1027/2064] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:56 2022] DELETE /identity/v3/users/245c0de3d5de42e3b33e261ab635fe95 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:57.499232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:57.500774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:57.528069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa134b5-598d-44c5-a108-13d172fe4beb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:57.599941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:57.600603 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:57.600892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:57.601711 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] PATH_INFO: `/v3/projects/5cbef8ed086d4e8c821483cdf2b4c607` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:57.603189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cbef8ed086d4e8c821483cdf2b4c607)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:57.605244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:57.640885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:57.665590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cbef8ed086d4e8c821483cdf2b4c607', 'request_id': 'req-4fe6c6c5-0195-454e-923c-18b1ae086010'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:57.672532 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cbef8ed086d4e8c821483cdf2b4c607', 'request_id': 'req-4fe6c6c5-0195-454e-923c-18b1ae086010'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:57.680209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cbef8ed086d4e8c821483cdf2b4c607', 'request_id': 'req-4fe6c6c5-0195-454e-923c-18b1ae086010'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:57.703492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] The token cache is being invalidate because project 5cbef8ed086d4e8c821483cdf2b4c607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:57.705131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:57.706844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:57.714206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1038/2065] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:57 2022] DELETE /identity/v3/projects/5cbef8ed086d4e8c821483cdf2b4c607 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:57.715065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d81b9-cad2-4aed-8565-b6f746b370ee admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:57.716607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a63d81b9-cad2-4aed-8565-b6f746b370ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:57.735417 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d81b9-cad2-4aed-8565-b6f746b370ee admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:57.797227 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c29a2-21e7-4425-8c99-ba5040b0517f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:57.797888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c29a2-21e7-4425-8c99-ba5040b0517f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:57.798185 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c29a2-21e7-4425-8c99-ba5040b0517f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:57.798459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c29a2-21e7-4425-8c99-ba5040b0517f admin admin] PATH_INFO: `/v3/users/b0e27636ad904f378e3dc98be0b1d957` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:57.799762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c29a2-21e7-4425-8c99-ba5040b0517f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0e27636ad904f378e3dc98be0b1d957)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:57.801627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c29a2-21e7-4425-8c99-ba5040b0517f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:57.849601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:57.853359 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4c7c29a2-21e7-4425-8c99-ba5040b0517f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0e27636ad904f378e3dc98be0b1d957'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:57.863319 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4c7c29a2-21e7-4425-8c99-ba5040b0517f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0e27636ad904f378e3dc98be0b1d957'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:57.864130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4c7c29a2-21e7-4425-8c99-ba5040b0517f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0e27636ad904f378e3dc98be0b1d957'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:57.864744 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4c7c29a2-21e7-4425-8c99-ba5040b0517f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0e27636ad904f378e3dc98be0b1d957'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:57.877896 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-4c7c29a2-21e7-4425-8c99-ba5040b0517f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0e27636ad904f378e3dc98be0b1d957'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:57.890951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1028/2066] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:57 2022] DELETE /identity/v3/users/b0e27636ad904f378e3dc98be0b1d957 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:58.196313 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:58.200908 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:58.226323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6c6c5-0195-454e-923c-18b1ae086010 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:58.285653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d83f0-9177-43bc-b659-22cb38dc476e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:58.286464 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d83f0-9177-43bc-b659-22cb38dc476e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:58.286632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d83f0-9177-43bc-b659-22cb38dc476e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:58.286939 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d83f0-9177-43bc-b659-22cb38dc476e admin admin] PATH_INFO: `/v3/projects/5618e2b6aacd4134b9ae78079ef614e4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:58.288122 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d83f0-9177-43bc-b659-22cb38dc476e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5618e2b6aacd4134b9ae78079ef614e4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:58.290253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d83f0-9177-43bc-b659-22cb38dc476e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:58.339561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b50d83f0-9177-43bc-b659-22cb38dc476e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5618e2b6aacd4134b9ae78079ef614e4', 'request_id': 'req-b50d83f0-9177-43bc-b659-22cb38dc476e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:58.346244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b50d83f0-9177-43bc-b659-22cb38dc476e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5618e2b6aacd4134b9ae78079ef614e4', 'request_id': 'req-b50d83f0-9177-43bc-b659-22cb38dc476e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:58.352424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b50d83f0-9177-43bc-b659-22cb38dc476e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5618e2b6aacd4134b9ae78079ef614e4', 'request_id': 'req-b50d83f0-9177-43bc-b659-22cb38dc476e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:58.366048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b50d83f0-9177-43bc-b659-22cb38dc476e admin admin] The token cache is being invalidate because project 5618e2b6aacd4134b9ae78079ef614e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:58.366971 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b50d83f0-9177-43bc-b659-22cb38dc476e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:58.367220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b50d83f0-9177-43bc-b659-22cb38dc476e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:58.368754 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1039/2067] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:58 2022] DELETE /identity/v3/projects/5618e2b6aacd4134b9ae78079ef614e4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:58.374801 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ed41f-8c34-4fab-bb21-f1170f1a5c13 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:58.375125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ed41f-8c34-4fab-bb21-f1170f1a5c13 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:58.375408 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ed41f-8c34-4fab-bb21-f1170f1a5c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:58.396069 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-186ed41f-8c34-4fab-bb21-f1170f1a5c13 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:58.397294 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-186ed41f-8c34-4fab-bb21-f1170f1a5c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:58.487589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1029/2068] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:58.496827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4db195-eac7-49ca-8a24-73bfbeb66bcb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:58.497112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4db195-eac7-49ca-8a24-73bfbeb66bcb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:58.497394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4db195-eac7-49ca-8a24-73bfbeb66bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:58.519369 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ce4db195-eac7-49ca-8a24-73bfbeb66bcb None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:58.520693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ce4db195-eac7-49ca-8a24-73bfbeb66bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:58.534701 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1040/2069] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:58.535062 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ed41f-8c34-4fab-bb21-f1170f1a5c13 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:58.549593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ed41f-8c34-4fab-bb21-f1170f1a5c13 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:58.554508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36ee16-7692-420a-b10b-8d04d34591a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:58.555166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36ee16-7692-420a-b10b-8d04d34591a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:58.555451 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36ee16-7692-420a-b10b-8d04d34591a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:58.555727 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36ee16-7692-420a-b10b-8d04d34591a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:58.556439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36ee16-7692-420a-b10b-8d04d34591a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:58.556746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36ee16-7692-420a-b10b-8d04d34591a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:58.558372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36ee16-7692-420a-b10b-8d04d34591a0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:58.571389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1030/2070] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:15:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:15:58.575683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4db195-eac7-49ca-8a24-73bfbeb66bcb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:58.586204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4db195-eac7-49ca-8a24-73bfbeb66bcb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:58.591315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ca2c4-f41a-49f7-9a51-db28a44119fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:58.591974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ca2c4-f41a-49f7-9a51-db28a44119fc admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:58.592356 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ca2c4-f41a-49f7-9a51-db28a44119fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:58.592781 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ca2c4-f41a-49f7-9a51-db28a44119fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:58.593585 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ca2c4-f41a-49f7-9a51-db28a44119fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:58.595663 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ca2c4-f41a-49f7-9a51-db28a44119fc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:58.627815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1041/2071] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:15:58 2022] POST /identity/v3/projects => generated 374 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:15:58.629489 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36ee16-7692-420a-b10b-8d04d34591a0 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:58.640417 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36ee16-7692-420a-b10b-8d04d34591a0 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:58.697583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab25e9b-70a4-4f2d-99ec-0555f106bf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:58.698206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab25e9b-70a4-4f2d-99ec-0555f106bf44 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:58.698548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab25e9b-70a4-4f2d-99ec-0555f106bf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:58.698863 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab25e9b-70a4-4f2d-99ec-0555f106bf44 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:58.699565 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab25e9b-70a4-4f2d-99ec-0555f106bf44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:58.701601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab25e9b-70a4-4f2d-99ec-0555f106bf44 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:58.723833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1031/2072] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:15:58 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:15:58.728069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ca2c4-f41a-49f7-9a51-db28a44119fc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:58.738885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ca2c4-f41a-49f7-9a51-db28a44119fc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:58.744199 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02121412-3a48-44b5-b330-c5cbda7daeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:58.744840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02121412-3a48-44b5-b330-c5cbda7daeba admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:58.745112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02121412-3a48-44b5-b330-c5cbda7daeba admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:58.745449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02121412-3a48-44b5-b330-c5cbda7daeba admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:58.748315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02121412-3a48-44b5-b330-c5cbda7daeba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:58.749907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02121412-3a48-44b5-b330-c5cbda7daeba admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:58.761914 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1042/2073] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:58 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:58.765934 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab25e9b-70a4-4f2d-99ec-0555f106bf44 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:58.776215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab25e9b-70a4-4f2d-99ec-0555f106bf44 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:58.781046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495364b-1ad2-4ee7-b4ef-2c9516ada826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:58.781658 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495364b-1ad2-4ee7-b4ef-2c9516ada826 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:58.782694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495364b-1ad2-4ee7-b4ef-2c9516ada826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:58.782694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495364b-1ad2-4ee7-b4ef-2c9516ada826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:58.782976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495364b-1ad2-4ee7-b4ef-2c9516ada826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:58.784580 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495364b-1ad2-4ee7-b4ef-2c9516ada826 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:58.796316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1032/2074] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:15:58 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:15:58.800278 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02121412-3a48-44b5-b330-c5cbda7daeba admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:58.807195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:58.811141 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02121412-3a48-44b5-b330-c5cbda7daeba admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:58.817079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3eb66-9f1f-46a3-8796-d0ab073f1641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:58.818014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3eb66-9f1f-46a3-8796-d0ab073f1641 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:58.818358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3eb66-9f1f-46a3-8796-d0ab073f1641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:58.818856 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3eb66-9f1f-46a3-8796-d0ab073f1641 admin admin] PATH_INFO: `/v3/projects/db3fbed0dc004863b8bd4007553f9354/users/0e3f1f7835fc422e9038dab0df2cae50/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:58.833981 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3eb66-9f1f-46a3-8796-d0ab073f1641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3fbed0dc004863b8bd4007553f9354, user_id=0e3f1f7835fc422e9038dab0df2cae50, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:58.837222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3eb66-9f1f-46a3-8796-d0ab073f1641 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:58.850527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1043/2075] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:15:58 2022] PUT /identity/v3/projects/db3fbed0dc004863b8bd4007553f9354/users/0e3f1f7835fc422e9038dab0df2cae50/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:58.861287 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843fcb1-f2e8-4100-ab3e-22027e9fe7d5 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:58.861840 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843fcb1-f2e8-4100-ab3e-22027e9fe7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:58.861840 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843fcb1-f2e8-4100-ab3e-22027e9fe7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:58.890629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0843fcb1-f2e8-4100-ab3e-22027e9fe7d5 None None] MFA Rules not processed for user `0e3f1f7835fc422e9038dab0df2cae50`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:58.891925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0843fcb1-f2e8-4100-ab3e-22027e9fe7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:58.904945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:58.964028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1033/2076] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:58.975299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3eb66-9f1f-46a3-8796-d0ab073f1641 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:58.977731 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-24a3eb66-9f1f-46a3-8796-d0ab073f1641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:59.038833 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3eb66-9f1f-46a3-8796-d0ab073f1641 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:59.043272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388eed9-d83a-44c3-87c5-8e227755f703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:59.043896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388eed9-d83a-44c3-87c5-8e227755f703 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:59.044180 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388eed9-d83a-44c3-87c5-8e227755f703 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:59.044457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388eed9-d83a-44c3-87c5-8e227755f703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:59.053202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388eed9-d83a-44c3-87c5-8e227755f703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:59.054700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388eed9-d83a-44c3-87c5-8e227755f703 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:59.063978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1044/2077] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:59.092503 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843fcb1-f2e8-4100-ab3e-22027e9fe7d5 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:59.158546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843fcb1-f2e8-4100-ab3e-22027e9fe7d5 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:59.164378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78686f-3e25-4065-ab0c-3bc0cc4e6a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:59.165090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78686f-3e25-4065-ab0c-3bc0cc4e6a90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:59.165372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78686f-3e25-4065-ab0c-3bc0cc4e6a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:59.165652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78686f-3e25-4065-ab0c-3bc0cc4e6a90 admin admin] PATH_INFO: `/v3/users/0e3f1f7835fc422e9038dab0df2cae50` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:59.166946 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78686f-3e25-4065-ab0c-3bc0cc4e6a90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e3f1f7835fc422e9038dab0df2cae50)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:59.169024 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78686f-3e25-4065-ab0c-3bc0cc4e6a90 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:59.218777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8d78686f-3e25-4065-ab0c-3bc0cc4e6a90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e3f1f7835fc422e9038dab0df2cae50'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.226846 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8d78686f-3e25-4065-ab0c-3bc0cc4e6a90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e3f1f7835fc422e9038dab0df2cae50'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.227581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8d78686f-3e25-4065-ab0c-3bc0cc4e6a90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e3f1f7835fc422e9038dab0df2cae50'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.228220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8d78686f-3e25-4065-ab0c-3bc0cc4e6a90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e3f1f7835fc422e9038dab0df2cae50'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.240795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8d78686f-3e25-4065-ab0c-3bc0cc4e6a90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e3f1f7835fc422e9038dab0df2cae50'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.253593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1034/2078] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:15:59 2022] DELETE /identity/v3/users/0e3f1f7835fc422e9038dab0df2cae50 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:59.258612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314430e-b199-4de9-83f6-2ddd9d1cc6f5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:59.258937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314430e-b199-4de9-83f6-2ddd9d1cc6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:59.259213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4314430e-b199-4de9-83f6-2ddd9d1cc6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:59.279291 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4314430e-b199-4de9-83f6-2ddd9d1cc6f5 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:59.280648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4314430e-b199-4de9-83f6-2ddd9d1cc6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:59.347104 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1045/2079] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:59.356558 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78686f-3e25-4065-ab0c-3bc0cc4e6a90 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:59.357972 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8d78686f-3e25-4065-ab0c-3bc0cc4e6a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:59.404222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:59.423455 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78686f-3e25-4065-ab0c-3bc0cc4e6a90 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:59.427056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6ba903-ec88-48e4-9ca3-dfa37000f8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:59.427675 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ba903-ec88-48e4-9ca3-dfa37000f8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:59.427953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ba903-ec88-48e4-9ca3-dfa37000f8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:59.428222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6ba903-ec88-48e4-9ca3-dfa37000f8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:59.437937 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ba903-ec88-48e4-9ca3-dfa37000f8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:59.439407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6ba903-ec88-48e4-9ca3-dfa37000f8a0 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:59.449361 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1035/2080] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:15:59.727827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314430e-b199-4de9-83f6-2ddd9d1cc6f5 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:59.729351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4314430e-b199-4de9-83f6-2ddd9d1cc6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:59.743452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4314430e-b199-4de9-83f6-2ddd9d1cc6f5 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:59.749086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:59.750003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:59.750380 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:59.750851 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2 admin admin] PATH_INFO: `/v3/projects/db3fbed0dc004863b8bd4007553f9354` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:59.752028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db3fbed0dc004863b8bd4007553f9354)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:59.753934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:15:59.809208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db3fbed0dc004863b8bd4007553f9354', 'request_id': 'req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.815442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db3fbed0dc004863b8bd4007553f9354', 'request_id': 'req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.821497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db3fbed0dc004863b8bd4007553f9354', 'request_id': 'req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.835600 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2 admin admin] The token cache is being invalidate because project db3fbed0dc004863b8bd4007553f9354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:15:59.835893 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.836520 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ba5aad83-0de5-4fbd-8d78-e2cd004026d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:15:59.838432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1046/2081] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:15:59 2022] DELETE /identity/v3/projects/db3fbed0dc004863b8bd4007553f9354 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:15:59.847828 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654d511-6883-4e1e-8a86-d3c4e5c9d03b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:59.848114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654d511-6883-4e1e-8a86-d3c4e5c9d03b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:59.848390 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654d511-6883-4e1e-8a86-d3c4e5c9d03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:59.868744 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-0654d511-6883-4e1e-8a86-d3c4e5c9d03b None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:59.869947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0654d511-6883-4e1e-8a86-d3c4e5c9d03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:59.929862 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1036/2082] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:59.937562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac70198-4101-4663-9781-1ac08562c924 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:59.937853 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac70198-4101-4663-9781-1ac08562c924 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:59.938140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac70198-4101-4663-9781-1ac08562c924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:59.952455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:15:59.959074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2ac70198-4101-4663-9781-1ac08562c924 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:15:59.960940 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2ac70198-4101-4663-9781-1ac08562c924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:15:59.969588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1047/2083] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:15:59.973868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654d511-6883-4e1e-8a86-d3c4e5c9d03b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:15:59.984347 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654d511-6883-4e1e-8a86-d3c4e5c9d03b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:15:59.988953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbbd35-ebcc-44e7-9a34-cf7d72faabbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:15:59.989559 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbbd35-ebcc-44e7-9a34-cf7d72faabbb admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:15:59.989840 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbbd35-ebcc-44e7-9a34-cf7d72faabbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:15:59.990114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbbd35-ebcc-44e7-9a34-cf7d72faabbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:15:59.990842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbbd35-ebcc-44e7-9a34-cf7d72faabbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:15:59.991149 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbbd35-ebcc-44e7-9a34-cf7d72faabbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:15:59.993419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbbd35-ebcc-44e7-9a34-cf7d72faabbb admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:00.005305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1037/2084] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:15:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:16:00.009619 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac70198-4101-4663-9781-1ac08562c924 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:00.018895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac70198-4101-4663-9781-1ac08562c924 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:00.023595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13a02d-9e6b-472c-9592-a1378742e93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:00.024200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13a02d-9e6b-472c-9592-a1378742e93a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:00.024483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13a02d-9e6b-472c-9592-a1378742e93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:00.024758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13a02d-9e6b-472c-9592-a1378742e93a admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:00.025436 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13a02d-9e6b-472c-9592-a1378742e93a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:00.027384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13a02d-9e6b-472c-9592-a1378742e93a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:00.055153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1048/2085] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:16:00 2022] POST /identity/v3/projects => generated 382 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:16:00.059312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbbd35-ebcc-44e7-9a34-cf7d72faabbb admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:00.069399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbbd35-ebcc-44e7-9a34-cf7d72faabbb admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:00.128376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29337b08-46a2-40b9-b880-858b01c65fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:00.129082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29337b08-46a2-40b9-b880-858b01c65fbd admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:00.129336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29337b08-46a2-40b9-b880-858b01c65fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:00.129613 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29337b08-46a2-40b9-b880-858b01c65fbd admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:00.130337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29337b08-46a2-40b9-b880-858b01c65fbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:00.132292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29337b08-46a2-40b9-b880-858b01c65fbd admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:00.154032 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1038/2086] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:16:00 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:16:00.158192 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13a02d-9e6b-472c-9592-a1378742e93a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:00.169257 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13a02d-9e6b-472c-9592-a1378742e93a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:00.173906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432c40c-e778-48c7-a97f-7242967843e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:00.174521 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432c40c-e778-48c7-a97f-7242967843e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:00.174839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432c40c-e778-48c7-a97f-7242967843e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:00.175119 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432c40c-e778-48c7-a97f-7242967843e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:00.175818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432c40c-e778-48c7-a97f-7242967843e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:00.177415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f432c40c-e778-48c7-a97f-7242967843e5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:00.190428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1049/2087] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:00 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:00.192604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29337b08-46a2-40b9-b880-858b01c65fbd admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:00.203413 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29337b08-46a2-40b9-b880-858b01c65fbd admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:00.209119 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f6a5e-e150-443b-8b4a-1c6595b5568f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:00.209744 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f6a5e-e150-443b-8b4a-1c6595b5568f admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:00.210829 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f6a5e-e150-443b-8b4a-1c6595b5568f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:00.211114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f6a5e-e150-443b-8b4a-1c6595b5568f admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:00.211827 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f6a5e-e150-443b-8b4a-1c6595b5568f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:00.213426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f6a5e-e150-443b-8b4a-1c6595b5568f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:00.224901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1039/2088] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:00 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:00.229741 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432c40c-e778-48c7-a97f-7242967843e5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:00.240460 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432c40c-e778-48c7-a97f-7242967843e5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:00.245691 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209f015-ab7d-4b4b-acb9-dde017f41416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:00.246374 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209f015-ab7d-4b4b-acb9-dde017f41416 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:00.246656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209f015-ab7d-4b4b-acb9-dde017f41416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:00.246987 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209f015-ab7d-4b4b-acb9-dde017f41416 admin admin] PATH_INFO: `/v3/projects/060a7c219492407581c39806cc5726a3/users/b507df75a6564522bf6746587d081a03/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:00.264253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209f015-ab7d-4b4b-acb9-dde017f41416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=060a7c219492407581c39806cc5726a3, user_id=b507df75a6564522bf6746587d081a03, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:00.267191 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209f015-ab7d-4b4b-acb9-dde017f41416 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:00.284099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1050/2089] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:00 2022] PUT /identity/v3/projects/060a7c219492407581c39806cc5726a3/users/b507df75a6564522bf6746587d081a03/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:00.290039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10d588-fc6a-42e0-872b-3e89d411f54a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:00.290357 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10d588-fc6a-42e0-872b-3e89d411f54a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:00.290634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10d588-fc6a-42e0-872b-3e89d411f54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:00.310014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3d10d588-fc6a-42e0-872b-3e89d411f54a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:00.311291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3d10d588-fc6a-42e0-872b-3e89d411f54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:00.360911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:00.385902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1040/2090] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:00.399527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209f015-ab7d-4b4b-acb9-dde017f41416 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:00.401424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5209f015-ab7d-4b4b-acb9-dde017f41416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:00.472086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209f015-ab7d-4b4b-acb9-dde017f41416 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:00.475956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cd024-37bb-4e9b-b26f-608cd8419bf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:00.476662 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cd024-37bb-4e9b-b26f-608cd8419bf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:00.476945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cd024-37bb-4e9b-b26f-608cd8419bf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:00.477448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79cd024-37bb-4e9b-b26f-608cd8419bf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:00.493905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cd024-37bb-4e9b-b26f-608cd8419bf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:00.495405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79cd024-37bb-4e9b-b26f-608cd8419bf6 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:00.505801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1051/2091] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:01.296793 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5398c-9ce7-4f7a-8efc-45200107da71 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:01.297326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5398c-9ce7-4f7a-8efc-45200107da71 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:01.297326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5398c-9ce7-4f7a-8efc-45200107da71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:01.328691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c2d5398c-9ce7-4f7a-8efc-45200107da71 None None] MFA Rules not processed for user `b507df75a6564522bf6746587d081a03`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:01.329941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c2d5398c-9ce7-4f7a-8efc-45200107da71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:01.401619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1041/2092] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:01.412949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cd024-37bb-4e9b-b26f-608cd8419bf6 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:01.416511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c79cd024-37bb-4e9b-b26f-608cd8419bf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:01.488820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79cd024-37bb-4e9b-b26f-608cd8419bf6 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:01.492972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41ae08-8bec-42bb-adbc-2f18a73bdcc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:01.493692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41ae08-8bec-42bb-adbc-2f18a73bdcc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:01.494169 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41ae08-8bec-42bb-adbc-2f18a73bdcc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:01.494530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41ae08-8bec-42bb-adbc-2f18a73bdcc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:01.503965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41ae08-8bec-42bb-adbc-2f18a73bdcc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:01.505557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41ae08-8bec-42bb-adbc-2f18a73bdcc3 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:01.515804 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1052/2093] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:05.076456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55a3bf-b370-465d-b9e1-791c6f504c0f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:05.076953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55a3bf-b370-465d-b9e1-791c6f504c0f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:05.077045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55a3bf-b370-465d-b9e1-791c6f504c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:05.097045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3e55a3bf-b370-465d-b9e1-791c6f504c0f None None] MFA Rules not processed for user `b507df75a6564522bf6746587d081a03`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:05.098283 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3e55a3bf-b370-465d-b9e1-791c6f504c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:05.110693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1042/2094] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:06.187737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41ae08-8bec-42bb-adbc-2f18a73bdcc3 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:06.189282 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5e41ae08-8bec-42bb-adbc-2f18a73bdcc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:06.243841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41ae08-8bec-42bb-adbc-2f18a73bdcc3 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:06.259600 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76937ffc-bd8d-49a8-8e2d-02633713e1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:06.260397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76937ffc-bd8d-49a8-8e2d-02633713e1f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:06.260921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76937ffc-bd8d-49a8-8e2d-02633713e1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:06.261350 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76937ffc-bd8d-49a8-8e2d-02633713e1f2 admin admin] PATH_INFO: `/v3/users/125e197e3bb549f7a113a91d0432f9df` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:06.263175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76937ffc-bd8d-49a8-8e2d-02633713e1f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=125e197e3bb549f7a113a91d0432f9df)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:06.266011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76937ffc-bd8d-49a8-8e2d-02633713e1f2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:06.358107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:06.362768 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-76937ffc-bd8d-49a8-8e2d-02633713e1f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '125e197e3bb549f7a113a91d0432f9df'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:06.364113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-76937ffc-bd8d-49a8-8e2d-02633713e1f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '125e197e3bb549f7a113a91d0432f9df'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:06.373634 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-76937ffc-bd8d-49a8-8e2d-02633713e1f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '125e197e3bb549f7a113a91d0432f9df'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:06.375086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-76937ffc-bd8d-49a8-8e2d-02633713e1f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '125e197e3bb549f7a113a91d0432f9df'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:06.388506 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-76937ffc-bd8d-49a8-8e2d-02633713e1f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '125e197e3bb549f7a113a91d0432f9df'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:06.404966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1053/2095] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:16:06 2022] DELETE /identity/v3/users/125e197e3bb549f7a113a91d0432f9df => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:06.409499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08082b7-5158-48e3-8ab9-9979e3ffdb38 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:06.410017 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08082b7-5158-48e3-8ab9-9979e3ffdb38 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:06.410179 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08082b7-5158-48e3-8ab9-9979e3ffdb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:06.447671 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b08082b7-5158-48e3-8ab9-9979e3ffdb38 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:06.449070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b08082b7-5158-48e3-8ab9-9979e3ffdb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:06.528833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1043/2096] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:06.539120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76937ffc-bd8d-49a8-8e2d-02633713e1f2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:06.540783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-76937ffc-bd8d-49a8-8e2d-02633713e1f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:06.604910 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76937ffc-bd8d-49a8-8e2d-02633713e1f2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:06.609849 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3891ab9-a40e-4556-9c49-34f523475fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:06.610518 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3891ab9-a40e-4556-9c49-34f523475fea service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:06.610865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3891ab9-a40e-4556-9c49-34f523475fea service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:06.611150 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3891ab9-a40e-4556-9c49-34f523475fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:06.621263 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3891ab9-a40e-4556-9c49-34f523475fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:06.622800 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3891ab9-a40e-4556-9c49-34f523475fea service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:06.633725 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1054/2097] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:06.936621 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08082b7-5158-48e3-8ab9-9979e3ffdb38 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:06.938690 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b08082b7-5158-48e3-8ab9-9979e3ffdb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:06.955340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08082b7-5158-48e3-8ab9-9979e3ffdb38 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:06.961108 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f80e298-2cfb-44c7-9134-75389dbd1b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:06.961802 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80e298-2cfb-44c7-9134-75389dbd1b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:06.962089 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80e298-2cfb-44c7-9134-75389dbd1b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:06.962383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f80e298-2cfb-44c7-9134-75389dbd1b5f admin admin] PATH_INFO: `/v3/projects/371daf03994b41649d4edb279fab49ff` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:06.963757 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80e298-2cfb-44c7-9134-75389dbd1b5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=371daf03994b41649d4edb279fab49ff)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:06.965773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f80e298-2cfb-44c7-9134-75389dbd1b5f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:07.018880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2f80e298-2cfb-44c7-9134-75389dbd1b5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '371daf03994b41649d4edb279fab49ff', 'request_id': 'req-2f80e298-2cfb-44c7-9134-75389dbd1b5f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.025812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2f80e298-2cfb-44c7-9134-75389dbd1b5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '371daf03994b41649d4edb279fab49ff', 'request_id': 'req-2f80e298-2cfb-44c7-9134-75389dbd1b5f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.034129 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2f80e298-2cfb-44c7-9134-75389dbd1b5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '371daf03994b41649d4edb279fab49ff', 'request_id': 'req-2f80e298-2cfb-44c7-9134-75389dbd1b5f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.036647 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:07.050999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2f80e298-2cfb-44c7-9134-75389dbd1b5f admin admin] The token cache is being invalidate because project 371daf03994b41649d4edb279fab49ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:07.051986 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2f80e298-2cfb-44c7-9134-75389dbd1b5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.052649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2f80e298-2cfb-44c7-9134-75389dbd1b5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.055058 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1044/2098] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:16:06 2022] DELETE /identity/v3/projects/371daf03994b41649d4edb279fab49ff => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:07.380604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3891ab9-a40e-4556-9c49-34f523475fea service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:07.382091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b3891ab9-a40e-4556-9c49-34f523475fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:07.395568 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3891ab9-a40e-4556-9c49-34f523475fea service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:07.453555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d5c1f-c561-4365-bf56-53ec59b95a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:07.454222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d5c1f-c561-4365-bf56-53ec59b95a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:07.454524 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d5c1f-c561-4365-bf56-53ec59b95a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:07.454847 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d5c1f-c561-4365-bf56-53ec59b95a57 admin admin] PATH_INFO: `/v3/users/855168ad1d974de48904e33531b5a219` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:07.456038 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d5c1f-c561-4365-bf56-53ec59b95a57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=855168ad1d974de48904e33531b5a219)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:07.458227 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d5c1f-c561-4365-bf56-53ec59b95a57 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:07.512015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dd9d5c1f-c561-4365-bf56-53ec59b95a57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '855168ad1d974de48904e33531b5a219'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.512676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dd9d5c1f-c561-4365-bf56-53ec59b95a57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '855168ad1d974de48904e33531b5a219'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.521500 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dd9d5c1f-c561-4365-bf56-53ec59b95a57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '855168ad1d974de48904e33531b5a219'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.522517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dd9d5c1f-c561-4365-bf56-53ec59b95a57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '855168ad1d974de48904e33531b5a219'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.531888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:07.538330 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dd9d5c1f-c561-4365-bf56-53ec59b95a57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '855168ad1d974de48904e33531b5a219'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:07.554643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1055/2099] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:16:07 2022] DELETE /identity/v3/users/855168ad1d974de48904e33531b5a219 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:07.560819 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817559b0-c046-4cb2-a628-67e8d0554dde None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:07.561612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817559b0-c046-4cb2-a628-67e8d0554dde None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:07.561612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817559b0-c046-4cb2-a628-67e8d0554dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:07.581184 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-817559b0-c046-4cb2-a628-67e8d0554dde None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:07.582728 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-817559b0-c046-4cb2-a628-67e8d0554dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:07.644660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1045/2100] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:07.655407 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d5c1f-c561-4365-bf56-53ec59b95a57 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:07.657112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dd9d5c1f-c561-4365-bf56-53ec59b95a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:07.736069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d5c1f-c561-4365-bf56-53ec59b95a57 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:07.740578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596f24b-d10e-4fda-ad3e-515d4be8f1dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:07.741366 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596f24b-d10e-4fda-ad3e-515d4be8f1dc service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:07.741801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596f24b-d10e-4fda-ad3e-515d4be8f1dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:07.742433 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596f24b-d10e-4fda-ad3e-515d4be8f1dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:07.755779 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f596f24b-d10e-4fda-ad3e-515d4be8f1dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:07.757357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f596f24b-d10e-4fda-ad3e-515d4be8f1dc service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:07.770864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1056/2101] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:16:08.148171 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817559b0-c046-4cb2-a628-67e8d0554dde None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:08.150111 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-817559b0-c046-4cb2-a628-67e8d0554dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:08.177103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817559b0-c046-4cb2-a628-67e8d0554dde None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:08.217348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0573ad8-d00d-4261-8acf-38943e06ce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:08.220638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0573ad8-d00d-4261-8acf-38943e06ce59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.220974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0573ad8-d00d-4261-8acf-38943e06ce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.228656 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0573ad8-d00d-4261-8acf-38943e06ce59 admin admin] PATH_INFO: `/v3/users/dbdaeedf7e494af7bafb1004fc6f23dc` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.235053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0573ad8-d00d-4261-8acf-38943e06ce59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbdaeedf7e494af7bafb1004fc6f23dc)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:08.237263 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0573ad8-d00d-4261-8acf-38943e06ce59 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:08.250352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596f24b-d10e-4fda-ad3e-515d4be8f1dc service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:08.252904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f596f24b-d10e-4fda-ad3e-515d4be8f1dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:08.274131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596f24b-d10e-4fda-ad3e-515d4be8f1dc service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:08.285511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b46e1-0381-4707-b0bf-898df10e88f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:08.286431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b46e1-0381-4707-b0bf-898df10e88f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.286828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b46e1-0381-4707-b0bf-898df10e88f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.287253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b46e1-0381-4707-b0bf-898df10e88f3 admin admin] PATH_INFO: `/v3/projects/354d6793216e4f87813f786a08a9188b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.289483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b46e1-0381-4707-b0bf-898df10e88f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=354d6793216e4f87813f786a08a9188b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:08.291699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b46e1-0381-4707-b0bf-898df10e88f3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:08.378783 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e0573ad8-d00d-4261-8acf-38943e06ce59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbdaeedf7e494af7bafb1004fc6f23dc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.400634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e0573ad8-d00d-4261-8acf-38943e06ce59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbdaeedf7e494af7bafb1004fc6f23dc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.401370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e0573ad8-d00d-4261-8acf-38943e06ce59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbdaeedf7e494af7bafb1004fc6f23dc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.402366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e0573ad8-d00d-4261-8acf-38943e06ce59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbdaeedf7e494af7bafb1004fc6f23dc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.432403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e0573ad8-d00d-4261-8acf-38943e06ce59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbdaeedf7e494af7bafb1004fc6f23dc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.436042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a36b46e1-0381-4707-b0bf-898df10e88f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '354d6793216e4f87813f786a08a9188b', 'request_id': 'req-a36b46e1-0381-4707-b0bf-898df10e88f3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.451010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1046/2102] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:16:08 2022] DELETE /identity/v3/users/dbdaeedf7e494af7bafb1004fc6f23dc => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:08.455769 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ebf7e-4678-4935-bd6e-d97b70e28a84 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.456134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ebf7e-4678-4935-bd6e-d97b70e28a84 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.456684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ebf7e-4678-4935-bd6e-d97b70e28a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.463349 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a36b46e1-0381-4707-b0bf-898df10e88f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '354d6793216e4f87813f786a08a9188b', 'request_id': 'req-a36b46e1-0381-4707-b0bf-898df10e88f3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.471178 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a36b46e1-0381-4707-b0bf-898df10e88f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '354d6793216e4f87813f786a08a9188b', 'request_id': 'req-a36b46e1-0381-4707-b0bf-898df10e88f3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.482751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-501ebf7e-4678-4935-bd6e-d97b70e28a84 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:08.484580 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-501ebf7e-4678-4935-bd6e-d97b70e28a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:08.489357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a36b46e1-0381-4707-b0bf-898df10e88f3 admin admin] The token cache is being invalidate because project 354d6793216e4f87813f786a08a9188b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:08.489761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a36b46e1-0381-4707-b0bf-898df10e88f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.490506 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a36b46e1-0381-4707-b0bf-898df10e88f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:08.492984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1057/2103] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:16:08 2022] DELETE /identity/v3/projects/354d6793216e4f87813f786a08a9188b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:08.497330 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f74e2-6a0e-48da-82a5-144355f33a88 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.497881 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f74e2-6a0e-48da-82a5-144355f33a88 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.499462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f74e2-6a0e-48da-82a5-144355f33a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.555414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c04f74e2-6a0e-48da-82a5-144355f33a88 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:08.556940 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c04f74e2-6a0e-48da-82a5-144355f33a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:08.583678 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1047/2104] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:08.603951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ebf7e-4678-4935-bd6e-d97b70e28a84 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:08.606465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-501ebf7e-4678-4935-bd6e-d97b70e28a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:08.616980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:08.682704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1058/2105] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:08.685950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f379c60-f43a-4508-926f-fd342e3f82ec None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.686183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f379c60-f43a-4508-926f-fd342e3f82ec None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.686476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f379c60-f43a-4508-926f-fd342e3f82ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.697239 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ebf7e-4678-4935-bd6e-d97b70e28a84 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:08.701486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef44142-5e97-4487-b935-23fbbd222b5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:08.702131 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef44142-5e97-4487-b935-23fbbd222b5f service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.702754 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef44142-5e97-4487-b935-23fbbd222b5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.703064 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef44142-5e97-4487-b935-23fbbd222b5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.706872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4f379c60-f43a-4508-926f-fd342e3f82ec None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:08.708072 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4f379c60-f43a-4508-926f-fd342e3f82ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:08.720055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef44142-5e97-4487-b935-23fbbd222b5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:08.720232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1059/2106] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:08.720347 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef44142-5e97-4487-b935-23fbbd222b5f service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:08.721041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f379c60-f43a-4508-926f-fd342e3f82ec None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:08.725424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:08.731450 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1048/2107] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:08.731642 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f379c60-f43a-4508-926f-fd342e3f82ec None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:08.737741 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54eb4ba-510c-4c05-86e9-b94d6c58e8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:08.739126 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54eb4ba-510c-4c05-86e9-b94d6c58e8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.739841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54eb4ba-510c-4c05-86e9-b94d6c58e8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.740458 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54eb4ba-510c-4c05-86e9-b94d6c58e8f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.741271 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54eb4ba-510c-4c05-86e9-b94d6c58e8f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:16:08.741685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54eb4ba-510c-4c05-86e9-b94d6c58e8f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:08.743680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54eb4ba-510c-4c05-86e9-b94d6c58e8f6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:08.762634 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1060/2108] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:16:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:16:08.763964 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef44142-5e97-4487-b935-23fbbd222b5f service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:08.776843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef44142-5e97-4487-b935-23fbbd222b5f service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:08.781921 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c953049-5ebd-40b3-a911-e62a77db6f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:08.782542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953049-5ebd-40b3-a911-e62a77db6f1c admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.782883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953049-5ebd-40b3-a911-e62a77db6f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.783165 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953049-5ebd-40b3-a911-e62a77db6f1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.783978 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c953049-5ebd-40b3-a911-e62a77db6f1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:08.785826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c953049-5ebd-40b3-a911-e62a77db6f1c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:08.835861 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1049/2109] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:16:08 2022] POST /identity/v3/projects => generated 372 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:08.840496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54eb4ba-510c-4c05-86e9-b94d6c58e8f6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:08.856121 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54eb4ba-510c-4c05-86e9-b94d6c58e8f6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:08.957460 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e8433-a2bc-4ce1-b7a9-d033977b3fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:08.958185 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e8433-a2bc-4ce1-b7a9-d033977b3fdf admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:08.958476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e8433-a2bc-4ce1-b7a9-d033977b3fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:08.958859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e8433-a2bc-4ce1-b7a9-d033977b3fdf admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:08.960528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e8433-a2bc-4ce1-b7a9-d033977b3fdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:08.963081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e8433-a2bc-4ce1-b7a9-d033977b3fdf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:08.997894 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1061/2110] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:16:08 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:16:08.999078 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c953049-5ebd-40b3-a911-e62a77db6f1c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.016215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c953049-5ebd-40b3-a911-e62a77db6f1c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.034899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fdfbc-e290-4d5d-839a-625f9141105b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.035668 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fdfbc-e290-4d5d-839a-625f9141105b admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.036037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fdfbc-e290-4d5d-839a-625f9141105b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.036390 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fdfbc-e290-4d5d-839a-625f9141105b admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.037175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389fdfbc-e290-4d5d-839a-625f9141105b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.039493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389fdfbc-e290-4d5d-839a-625f9141105b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:09.062711 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1050/2111] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:08 2022] GET /identity/v3/roles => generated 1427 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:16:09.066281 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e8433-a2bc-4ce1-b7a9-d033977b3fdf admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.079839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e8433-a2bc-4ce1-b7a9-d033977b3fdf admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.088209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77299502-5a7e-42a9-a591-23428720541f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.105621 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77299502-5a7e-42a9-a591-23428720541f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.106055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77299502-5a7e-42a9-a591-23428720541f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.106433 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77299502-5a7e-42a9-a591-23428720541f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.113740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77299502-5a7e-42a9-a591-23428720541f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.115739 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77299502-5a7e-42a9-a591-23428720541f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:09.132835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1062/2112] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:09 2022] GET /identity/v3/roles => generated 1427 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:09.139058 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fdfbc-e290-4d5d-839a-625f9141105b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.162097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fdfbc-e290-4d5d-839a-625f9141105b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.169521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a2430-5ac7-4cb5-8cb5-d3fbbe4b0262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.170348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a2430-5ac7-4cb5-8cb5-d3fbbe4b0262 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.170782 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a2430-5ac7-4cb5-8cb5-d3fbbe4b0262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.171748 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a2430-5ac7-4cb5-8cb5-d3fbbe4b0262 admin admin] PATH_INFO: `/v3/projects/cc38647adf7e41beb5c47c01149972e0/users/55bb7887d98f458688cffbccf4a64b53/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.199459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a2430-5ac7-4cb5-8cb5-d3fbbe4b0262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc38647adf7e41beb5c47c01149972e0, user_id=55bb7887d98f458688cffbccf4a64b53, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.203062 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a2430-5ac7-4cb5-8cb5-d3fbbe4b0262 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:09.230783 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1051/2113] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:09 2022] PUT /identity/v3/projects/cc38647adf7e41beb5c47c01149972e0/users/55bb7887d98f458688cffbccf4a64b53/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:09.231818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77299502-5a7e-42a9-a591-23428720541f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.232836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2d1d7-182a-4d37-8bb7-746abf93a42d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.233204 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2d1d7-182a-4d37-8bb7-746abf93a42d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.233584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2d1d7-182a-4d37-8bb7-746abf93a42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.234070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-77299502-5a7e-42a9-a591-23428720541f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:09.249819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77299502-5a7e-42a9-a591-23428720541f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.258018 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-78a2d1d7-182a-4d37-8bb7-746abf93a42d None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:09.260143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-78a2d1d7-182a-4d37-8bb7-746abf93a42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:09.336793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.338758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.338758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.339072 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] PATH_INFO: `/v3/projects/8fd938728e0144f39e0ca66e08b0738d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.340711 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd938728e0144f39e0ca66e08b0738d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.342928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:09.359994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1052/2114] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:09.370700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2d1d7-182a-4d37-8bb7-746abf93a42d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.425530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd938728e0144f39e0ca66e08b0738d', 'request_id': 'req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:09.439789 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd938728e0144f39e0ca66e08b0738d', 'request_id': 'req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:09.450730 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd938728e0144f39e0ca66e08b0738d', 'request_id': 'req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:09.462544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a2d1d7-182a-4d37-8bb7-746abf93a42d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.463622 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:09.470203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:09.470738 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] The token cache is being invalidate because project 8fd938728e0144f39e0ca66e08b0738d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:09.471366 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:09.472733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:09.475791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1063/2115] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:16:09 2022] DELETE /identity/v3/projects/8fd938728e0144f39e0ca66e08b0738d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:16:09.480455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:09.481972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:09.534144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030135e-abdb-4d2e-9c00-d945a3523c41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.535526 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030135e-abdb-4d2e-9c00-d945a3523c41 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.535899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030135e-abdb-4d2e-9c00-d945a3523c41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.536219 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030135e-abdb-4d2e-9c00-d945a3523c41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.539733 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e030135e-abdb-4d2e-9c00-d945a3523c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:09.560041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39d160-dc32-4cdb-8f79-65f9a3b9b493 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:09.567820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062febf-569a-4370-84a0-82221f6bde4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:09.568564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062febf-569a-4370-84a0-82221f6bde4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:09.568928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062febf-569a-4370-84a0-82221f6bde4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:09.569816 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062febf-569a-4370-84a0-82221f6bde4f admin admin] PATH_INFO: `/v3/users/9ca77de723554cb09468830ce73c3be4` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:09.571309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062febf-569a-4370-84a0-82221f6bde4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca77de723554cb09468830ce73c3be4)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.573513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062febf-569a-4370-84a0-82221f6bde4f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:09.629564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8062febf-569a-4370-84a0-82221f6bde4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca77de723554cb09468830ce73c3be4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:09.630608 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8062febf-569a-4370-84a0-82221f6bde4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca77de723554cb09468830ce73c3be4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:09.641100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8062febf-569a-4370-84a0-82221f6bde4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca77de723554cb09468830ce73c3be4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:09.642427 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8062febf-569a-4370-84a0-82221f6bde4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca77de723554cb09468830ce73c3be4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:09.648349 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030135e-abdb-4d2e-9c00-d945a3523c41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:09.651137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030135e-abdb-4d2e-9c00-d945a3523c41 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:09.665001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1053/2116] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:16:09.666234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-8062febf-569a-4370-84a0-82221f6bde4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca77de723554cb09468830ce73c3be4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:09.683250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1064/2117] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:16:09 2022] DELETE /identity/v3/users/9ca77de723554cb09468830ce73c3be4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:16:10.259110 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030135e-abdb-4d2e-9c00-d945a3523c41 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:10.278960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030135e-abdb-4d2e-9c00-d945a3523c41 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:10.373576 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9be632-9bbe-4eb2-884f-756bf440f51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:10.374506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9be632-9bbe-4eb2-884f-756bf440f51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:10.374961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9be632-9bbe-4eb2-884f-756bf440f51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:10.375812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9be632-9bbe-4eb2-884f-756bf440f51a admin admin] PATH_INFO: `/v3/projects/8073ac3a235f478498080852b80ddbb3` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:10.376511 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9be632-9bbe-4eb2-884f-756bf440f51a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8073ac3a235f478498080852b80ddbb3)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:10.379383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9be632-9bbe-4eb2-884f-756bf440f51a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:10.448093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6a9be632-9bbe-4eb2-884f-756bf440f51a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8073ac3a235f478498080852b80ddbb3', 'request_id': 'req-6a9be632-9bbe-4eb2-884f-756bf440f51a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:10.456110 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6a9be632-9bbe-4eb2-884f-756bf440f51a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8073ac3a235f478498080852b80ddbb3', 'request_id': 'req-6a9be632-9bbe-4eb2-884f-756bf440f51a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:10.464128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:10.474247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6a9be632-9bbe-4eb2-884f-756bf440f51a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8073ac3a235f478498080852b80ddbb3', 'request_id': 'req-6a9be632-9bbe-4eb2-884f-756bf440f51a'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:10.503916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6a9be632-9bbe-4eb2-884f-756bf440f51a admin admin] The token cache is being invalidate because project 8073ac3a235f478498080852b80ddbb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:10.506400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6a9be632-9bbe-4eb2-884f-756bf440f51a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:10.507950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6a9be632-9bbe-4eb2-884f-756bf440f51a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:10.509875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1054/2118] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:16:10 2022] DELETE /identity/v3/projects/8073ac3a235f478498080852b80ddbb3 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:10.517089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e81450-53f5-4653-aa3f-333156ad2938 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:10.517579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e81450-53f5-4653-aa3f-333156ad2938 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:10.517690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e81450-53f5-4653-aa3f-333156ad2938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:10.563823 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-51e81450-53f5-4653-aa3f-333156ad2938 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:10.565293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-51e81450-53f5-4653-aa3f-333156ad2938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:10.646733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1065/2119] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:10.656226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143e14f-320e-419a-94b8-d899cba18caa None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:10.657348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143e14f-320e-419a-94b8-d899cba18caa None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:10.657724 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143e14f-320e-419a-94b8-d899cba18caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:10.678405 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f143e14f-320e-419a-94b8-d899cba18caa None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:10.680186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f143e14f-320e-419a-94b8-d899cba18caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:10.689696 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1055/2120] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:10.694073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e81450-53f5-4653-aa3f-333156ad2938 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:10.705505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e81450-53f5-4653-aa3f-333156ad2938 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:10.711040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7897db-3bca-4a3c-99d5-3c70fd1da9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:10.711798 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7897db-3bca-4a3c-99d5-3c70fd1da9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:10.711969 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7897db-3bca-4a3c-99d5-3c70fd1da9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:10.712237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7897db-3bca-4a3c-99d5-3c70fd1da9f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:10.712981 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7897db-3bca-4a3c-99d5-3c70fd1da9f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:16:10.713293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7897db-3bca-4a3c-99d5-3c70fd1da9f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:10.714949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7897db-3bca-4a3c-99d5-3c70fd1da9f1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:10.730080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1066/2121] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:16:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:16:10.735182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143e14f-320e-419a-94b8-d899cba18caa None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:10.747672 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143e14f-320e-419a-94b8-d899cba18caa None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:10.755671 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebef347-b03f-4c1d-a00d-de96949e6996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:10.756674 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebef347-b03f-4c1d-a00d-de96949e6996 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:10.757203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebef347-b03f-4c1d-a00d-de96949e6996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:10.758466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebef347-b03f-4c1d-a00d-de96949e6996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:10.759259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebef347-b03f-4c1d-a00d-de96949e6996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:10.761363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebef347-b03f-4c1d-a00d-de96949e6996 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:10.799983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1056/2122] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:16:10 2022] POST /identity/v3/projects => generated 390 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:10.802067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebef347-b03f-4c1d-a00d-de96949e6996 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:10.802494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ab0a9-1899-4fab-9b9a-55539b47bc7f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:10.802808 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ab0a9-1899-4fab-9b9a-55539b47bc7f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:10.803090 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ab0a9-1899-4fab-9b9a-55539b47bc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:10.812763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:10.838715 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-517ab0a9-1899-4fab-9b9a-55539b47bc7f None None] MFA Rules not processed for user `55bb7887d98f458688cffbccf4a64b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:10.840011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-517ab0a9-1899-4fab-9b9a-55539b47bc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:10.892836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebef347-b03f-4c1d-a00d-de96949e6996 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:10.901568 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61edf8-3980-4c5f-8598-8a8181dc2f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:10.902629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61edf8-3980-4c5f-8598-8a8181dc2f2a admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:10.904272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61edf8-3980-4c5f-8598-8a8181dc2f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:10.904833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61edf8-3980-4c5f-8598-8a8181dc2f2a admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:10.906359 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61edf8-3980-4c5f-8598-8a8181dc2f2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:10.910161 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61edf8-3980-4c5f-8598-8a8181dc2f2a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:10.918983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1067/2123] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:10.930630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ab0a9-1899-4fab-9b9a-55539b47bc7f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:10.932325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-517ab0a9-1899-4fab-9b9a-55539b47bc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:10.948291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1057/2124] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:16:10 2022] POST /identity/v3/users => generated 296 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:10.952191 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61edf8-3980-4c5f-8598-8a8181dc2f2a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:10.966780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61edf8-3980-4c5f-8598-8a8181dc2f2a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:10.972343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107db62-6f3d-4369-a315-ac6d36f5e992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:10.972976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107db62-6f3d-4369-a315-ac6d36f5e992 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:10.973259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107db62-6f3d-4369-a315-ac6d36f5e992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:10.973556 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107db62-6f3d-4369-a315-ac6d36f5e992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:10.974270 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107db62-6f3d-4369-a315-ac6d36f5e992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:10.976688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107db62-6f3d-4369-a315-ac6d36f5e992 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:10.988628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1058/2125] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:10 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:16:10.993092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107db62-6f3d-4369-a315-ac6d36f5e992 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:11.004265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107db62-6f3d-4369-a315-ac6d36f5e992 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:11.014785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f175716-0a85-4626-ad17-bee0669c26c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:11.015570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f175716-0a85-4626-ad17-bee0669c26c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:11.016087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f175716-0a85-4626-ad17-bee0669c26c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:11.016465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f175716-0a85-4626-ad17-bee0669c26c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.017289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f175716-0a85-4626-ad17-bee0669c26c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:11.019380 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f175716-0a85-4626-ad17-bee0669c26c1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:11.026601 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ab0a9-1899-4fab-9b9a-55539b47bc7f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:11.031554 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1685d-9bfb-4445-ae4e-b5ddecfeb22c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:11.032429 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1685d-9bfb-4445-ae4e-b5ddecfeb22c service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:11.032866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1685d-9bfb-4445-ae4e-b5ddecfeb22c service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:11.033297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1685d-9bfb-4445-ae4e-b5ddecfeb22c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.037427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1059/2126] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:10 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:16:11.038947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f175716-0a85-4626-ad17-bee0669c26c1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:11.048481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1685d-9bfb-4445-ae4e-b5ddecfeb22c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:11.050621 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1685d-9bfb-4445-ae4e-b5ddecfeb22c service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:11.052300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f175716-0a85-4626-ad17-bee0669c26c1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:11.058306 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1898a-dee7-42db-bfc3-749a153bf17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:11.059104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1898a-dee7-42db-bfc3-749a153bf17e admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:11.059387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1898a-dee7-42db-bfc3-749a153bf17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:11.059672 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1898a-dee7-42db-bfc3-749a153bf17e admin admin] PATH_INFO: `/v3/projects/0a4532a633f34dd08ebfd54ff5d6b566/users/7ce33137ab9a44d39fea3e9bd3def87c/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.075998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1068/2127] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:16:11.090157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1898a-dee7-42db-bfc3-749a153bf17e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a4532a633f34dd08ebfd54ff5d6b566, user_id=7ce33137ab9a44d39fea3e9bd3def87c, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:11.094295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1898a-dee7-42db-bfc3-749a153bf17e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:11.118295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1060/2128] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:11 2022] PUT /identity/v3/projects/0a4532a633f34dd08ebfd54ff5d6b566/users/7ce33137ab9a44d39fea3e9bd3def87c/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:11.129405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab03159-e769-4edf-92fb-1cf08954b4dd None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:11.130055 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab03159-e769-4edf-92fb-1cf08954b4dd None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:11.130435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab03159-e769-4edf-92fb-1cf08954b4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.169576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-eab03159-e769-4edf-92fb-1cf08954b4dd None None] MFA Rules not processed for user `7ce33137ab9a44d39fea3e9bd3def87c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:11.171202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-eab03159-e769-4edf-92fb-1cf08954b4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:11.312696 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1069/2129] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:11.320280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1898a-dee7-42db-bfc3-749a153bf17e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:11.408329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1898a-dee7-42db-bfc3-749a153bf17e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:11.414566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632e7b2-7e90-4c26-aa40-8433381ba654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:11.414916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632e7b2-7e90-4c26-aa40-8433381ba654 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:11.415178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632e7b2-7e90-4c26-aa40-8433381ba654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:11.415459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632e7b2-7e90-4c26-aa40-8433381ba654 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.416352 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632e7b2-7e90-4c26-aa40-8433381ba654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:11.418583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632e7b2-7e90-4c26-aa40-8433381ba654 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:11.425194 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:11.454706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1061/2130] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:16:11 2022] POST /identity/v3/projects => generated 388 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:11.457274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab03159-e769-4edf-92fb-1cf08954b4dd None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:11.476453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab03159-e769-4edf-92fb-1cf08954b4dd None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:11.508451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:11.552743 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc04e6-f17d-4f50-8d2f-da55c431de04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:11.555154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc04e6-f17d-4f50-8d2f-da55c431de04 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:11.556157 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc04e6-f17d-4f50-8d2f-da55c431de04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:11.556768 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc04e6-f17d-4f50-8d2f-da55c431de04 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.557704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc04e6-f17d-4f50-8d2f-da55c431de04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:11.559956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc04e6-f17d-4f50-8d2f-da55c431de04 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:11.593754 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1070/2131] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:16:11 2022] POST /identity/v3/users => generated 301 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:16:11.598426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632e7b2-7e90-4c26-aa40-8433381ba654 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:11.612230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632e7b2-7e90-4c26-aa40-8433381ba654 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:11.617689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc95f0-0077-466c-9bc8-f4b9ed3e50d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:11.618326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc95f0-0077-466c-9bc8-f4b9ed3e50d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:11.618616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc95f0-0077-466c-9bc8-f4b9ed3e50d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:11.618919 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc95f0-0077-466c-9bc8-f4b9ed3e50d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.619634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc95f0-0077-466c-9bc8-f4b9ed3e50d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:11.621270 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc95f0-0077-466c-9bc8-f4b9ed3e50d7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:11.638857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1062/2132] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:11 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:11.642902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc04e6-f17d-4f50-8d2f-da55c431de04 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:11.656981 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc04e6-f17d-4f50-8d2f-da55c431de04 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:11.663550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670d4a7-477b-49c7-8dde-397c306fc96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:11.664253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670d4a7-477b-49c7-8dde-397c306fc96b admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:11.664543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670d4a7-477b-49c7-8dde-397c306fc96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:11.664831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670d4a7-477b-49c7-8dde-397c306fc96b admin admin] PATH_INFO: `/v3/projects/7dec32f929004895ad2aa31eade1a226/users/06f4cc297e794edb913c3e1a77837515/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.690259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670d4a7-477b-49c7-8dde-397c306fc96b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dec32f929004895ad2aa31eade1a226, user_id=06f4cc297e794edb913c3e1a77837515, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:11.690259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670d4a7-477b-49c7-8dde-397c306fc96b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:11.710397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1071/2133] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:11 2022] PUT /identity/v3/projects/7dec32f929004895ad2aa31eade1a226/users/06f4cc297e794edb913c3e1a77837515/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:11.711881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc95f0-0077-466c-9bc8-f4b9ed3e50d7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:11.723246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc95f0-0077-466c-9bc8-f4b9ed3e50d7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:11.792279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f5858-2f1a-4c3d-8d8f-fb1516b4f154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:11.792906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f5858-2f1a-4c3d-8d8f-fb1516b4f154 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:11.793189 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f5858-2f1a-4c3d-8d8f-fb1516b4f154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:11.793467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f5858-2f1a-4c3d-8d8f-fb1516b4f154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.796729 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f5858-2f1a-4c3d-8d8f-fb1516b4f154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:11.796870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f5858-2f1a-4c3d-8d8f-fb1516b4f154 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:11.808399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1063/2134] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:11 2022] GET /identity/v3/roles => generated 1427 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:16:11.812835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670d4a7-477b-49c7-8dde-397c306fc96b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:11.823517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670d4a7-477b-49c7-8dde-397c306fc96b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:11.828137 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763a1d3-2f67-47d0-b2d7-4991ee1d3200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:11.828790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763a1d3-2f67-47d0-b2d7-4991ee1d3200 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:11.829069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763a1d3-2f67-47d0-b2d7-4991ee1d3200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:11.829344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763a1d3-2f67-47d0-b2d7-4991ee1d3200 admin admin] PATH_INFO: `/v3/projects/7dec32f929004895ad2aa31eade1a226/users/06f4cc297e794edb913c3e1a77837515/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.831627 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763a1d3-2f67-47d0-b2d7-4991ee1d3200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dec32f929004895ad2aa31eade1a226, user_id=06f4cc297e794edb913c3e1a77837515, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:11.834219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763a1d3-2f67-47d0-b2d7-4991ee1d3200 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:11.845257 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1072/2135] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:11 2022] PUT /identity/v3/projects/7dec32f929004895ad2aa31eade1a226/users/06f4cc297e794edb913c3e1a77837515/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:11.857259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ef8b1-3930-4da1-8815-a7f4e0daa9cd None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:11.857550 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ef8b1-3930-4da1-8815-a7f4e0daa9cd None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:11.857859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ef8b1-3930-4da1-8815-a7f4e0daa9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:11.894020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-569ef8b1-3930-4da1-8815-a7f4e0daa9cd None None] MFA Rules not processed for user `06f4cc297e794edb913c3e1a77837515`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:11.895292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-569ef8b1-3930-4da1-8815-a7f4e0daa9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:11.966375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1064/2136] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:11.976963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763a1d3-2f67-47d0-b2d7-4991ee1d3200 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:11.978742 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0763a1d3-2f67-47d0-b2d7-4991ee1d3200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:12.039072 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763a1d3-2f67-47d0-b2d7-4991ee1d3200 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:12.042782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee87ba8-257d-4957-9ef0-07d97f40a308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:12.043404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee87ba8-257d-4957-9ef0-07d97f40a308 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.043868 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee87ba8-257d-4957-9ef0-07d97f40a308 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.043868 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee87ba8-257d-4957-9ef0-07d97f40a308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.054037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee87ba8-257d-4957-9ef0-07d97f40a308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:12.055554 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee87ba8-257d-4957-9ef0-07d97f40a308 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:12.065353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1073/2137] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:12.211033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ef8b1-3930-4da1-8815-a7f4e0daa9cd None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:12.285610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ef8b1-3930-4da1-8815-a7f4e0daa9cd None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:12.292256 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad2429-43dd-4e8c-911e-489a453f260a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:12.292963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2429-43dd-4e8c-911e-489a453f260a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.293106 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2429-43dd-4e8c-911e-489a453f260a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.293299 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad2429-43dd-4e8c-911e-489a453f260a admin admin] PATH_INFO: `/v3/users/7ce33137ab9a44d39fea3e9bd3def87c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.294553 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad2429-43dd-4e8c-911e-489a453f260a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ce33137ab9a44d39fea3e9bd3def87c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:12.297378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:12.299065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad2429-43dd-4e8c-911e-489a453f260a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:12.394022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a5ad2429-43dd-4e8c-911e-489a453f260a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ce33137ab9a44d39fea3e9bd3def87c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:12.407178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a5ad2429-43dd-4e8c-911e-489a453f260a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ce33137ab9a44d39fea3e9bd3def87c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:12.408048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a5ad2429-43dd-4e8c-911e-489a453f260a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ce33137ab9a44d39fea3e9bd3def87c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:12.409610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a5ad2429-43dd-4e8c-911e-489a453f260a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ce33137ab9a44d39fea3e9bd3def87c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:12.430007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-a5ad2429-43dd-4e8c-911e-489a453f260a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ce33137ab9a44d39fea3e9bd3def87c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:12.472219 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1065/2138] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:16:12 2022] DELETE /identity/v3/users/7ce33137ab9a44d39fea3e9bd3def87c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:12.477272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090e062-7265-4964-bb91-b6ded56a27f9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.477538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090e062-7265-4964-bb91-b6ded56a27f9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.477808 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090e062-7265-4964-bb91-b6ded56a27f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.525654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6090e062-7265-4964-bb91-b6ded56a27f9 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:12.527547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6090e062-7265-4964-bb91-b6ded56a27f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:12.637431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1074/2139] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:12.640804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b6515-d397-423e-9c43-f844419847d4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.640804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b6515-d397-423e-9c43-f844419847d4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.641051 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b6515-d397-423e-9c43-f844419847d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.650505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090e062-7265-4964-bb91-b6ded56a27f9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:12.653364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6090e062-7265-4964-bb91-b6ded56a27f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:12.673228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-263b6515-d397-423e-9c43-f844419847d4 None None] MFA Rules not processed for user `55bb7887d98f458688cffbccf4a64b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:12.676222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-263b6515-d397-423e-9c43-f844419847d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:12.688362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:12.741713 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090e062-7265-4964-bb91-b6ded56a27f9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:12.747280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258f08e-932c-4017-b754-54f7dec1ebbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:12.748334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258f08e-932c-4017-b754-54f7dec1ebbb service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:12.748870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258f08e-932c-4017-b754-54f7dec1ebbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:12.749372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258f08e-932c-4017-b754-54f7dec1ebbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:12.756583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1066/2140] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:12.768607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258f08e-932c-4017-b754-54f7dec1ebbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:12.770126 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4258f08e-932c-4017-b754-54f7dec1ebbb service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:12.785309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1075/2141] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:13.156195 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b6515-d397-423e-9c43-f844419847d4 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:13.157739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-263b6515-d397-423e-9c43-f844419847d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:13.188436 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b6515-d397-423e-9c43-f844419847d4 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:13.199381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:13.200123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:13.200486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:13.200773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] PATH_INFO: `/v3/projects/0a4532a633f34dd08ebfd54ff5d6b566` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:13.202343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a4532a633f34dd08ebfd54ff5d6b566)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:13.204445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:13.267093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a4532a633f34dd08ebfd54ff5d6b566', 'request_id': 'req-25965ec6-9c0b-4263-9cbc-f670545ad211'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.274975 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a4532a633f34dd08ebfd54ff5d6b566', 'request_id': 'req-25965ec6-9c0b-4263-9cbc-f670545ad211'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.284131 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a4532a633f34dd08ebfd54ff5d6b566', 'request_id': 'req-25965ec6-9c0b-4263-9cbc-f670545ad211'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.298898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] The token cache is being invalidate because project 0a4532a633f34dd08ebfd54ff5d6b566 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:13.299196 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.300260 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.305327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1067/2142] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:16:13 2022] DELETE /identity/v3/projects/0a4532a633f34dd08ebfd54ff5d6b566 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:16:13.306388 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258f08e-932c-4017-b754-54f7dec1ebbb service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:13.318300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258f08e-932c-4017-b754-54f7dec1ebbb service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:13.382407 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b29e75-62a0-48d9-906c-79a6ff31c43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:13.383161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b29e75-62a0-48d9-906c-79a6ff31c43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:13.383440 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b29e75-62a0-48d9-906c-79a6ff31c43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:13.383714 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b29e75-62a0-48d9-906c-79a6ff31c43f admin admin] PATH_INFO: `/v3/users/06f4cc297e794edb913c3e1a77837515` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:13.384930 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b29e75-62a0-48d9-906c-79a6ff31c43f admin admin] RBAC: Authorizing `identity:delete_user(user_id=06f4cc297e794edb913c3e1a77837515)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:13.387037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b29e75-62a0-48d9-906c-79a6ff31c43f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:13.443109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-57b29e75-62a0-48d9-906c-79a6ff31c43f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06f4cc297e794edb913c3e1a77837515'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.443770 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-57b29e75-62a0-48d9-906c-79a6ff31c43f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06f4cc297e794edb913c3e1a77837515'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.451158 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-57b29e75-62a0-48d9-906c-79a6ff31c43f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06f4cc297e794edb913c3e1a77837515'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.452042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-57b29e75-62a0-48d9-906c-79a6ff31c43f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06f4cc297e794edb913c3e1a77837515'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.465772 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-57b29e75-62a0-48d9-906c-79a6ff31c43f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06f4cc297e794edb913c3e1a77837515'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:13.484822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1076/2143] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:16:13 2022] DELETE /identity/v3/users/06f4cc297e794edb913c3e1a77837515 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:13.832053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:13.833719 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:13.846684 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:13.866622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25965ec6-9c0b-4263-9cbc-f670545ad211 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:13.959316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:13.960695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:13.961162 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:13.961573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc admin admin] PATH_INFO: `/v3/projects/7dec32f929004895ad2aa31eade1a226` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:13.963210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dec32f929004895ad2aa31eade1a226)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:13.965143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:14.033340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dec32f929004895ad2aa31eade1a226', 'request_id': 'req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:14.040322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dec32f929004895ad2aa31eade1a226', 'request_id': 'req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:14.050375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dec32f929004895ad2aa31eade1a226', 'request_id': 'req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:14.069939 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc admin admin] The token cache is being invalidate because project 7dec32f929004895ad2aa31eade1a226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:14.070415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:14.071486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cbeca9d2-434b-403c-9d9c-7ad5c4e310bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:14.074238 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1068/2144] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:16:13 2022] DELETE /identity/v3/projects/7dec32f929004895ad2aa31eade1a226 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:14.081622 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9e19a-7148-4141-af22-d72444cb1bec None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.082753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9e19a-7148-4141-af22-d72444cb1bec None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.083232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9e19a-7148-4141-af22-d72444cb1bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.115026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-eaa9e19a-7148-4141-af22-d72444cb1bec None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:14.115026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-eaa9e19a-7148-4141-af22-d72444cb1bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:14.175335 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:14.214704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1077/2145] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:14.224259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e870f0-7b11-448e-96b3-e8189f692d8a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.224843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e870f0-7b11-448e-96b3-e8189f692d8a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.225706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e870f0-7b11-448e-96b3-e8189f692d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.267327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e6e870f0-7b11-448e-96b3-e8189f692d8a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:14.268857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e6e870f0-7b11-448e-96b3-e8189f692d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:14.278649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1069/2146] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:14.284220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9e19a-7148-4141-af22-d72444cb1bec None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:14.296016 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9e19a-7148-4141-af22-d72444cb1bec None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:14.301420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f72a5-1c7a-4a5c-905c-b3968cebc0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:14.302213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f72a5-1c7a-4a5c-905c-b3968cebc0db admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.302921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f72a5-1c7a-4a5c-905c-b3968cebc0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.303359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f72a5-1c7a-4a5c-905c-b3968cebc0db admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.304243 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f72a5-1c7a-4a5c-905c-b3968cebc0db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:16:14.304696 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f72a5-1c7a-4a5c-905c-b3968cebc0db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:14.306477 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f72a5-1c7a-4a5c-905c-b3968cebc0db admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:14.321778 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1078/2147] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:16:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:16:14.328465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e870f0-7b11-448e-96b3-e8189f692d8a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:14.344420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e870f0-7b11-448e-96b3-e8189f692d8a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:14.351414 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1161ddb-57bf-4d91-be3c-acd2b980925e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:14.352485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1161ddb-57bf-4d91-be3c-acd2b980925e admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.352982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1161ddb-57bf-4d91-be3c-acd2b980925e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.353465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1161ddb-57bf-4d91-be3c-acd2b980925e admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.354753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1161ddb-57bf-4d91-be3c-acd2b980925e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:14.358275 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1161ddb-57bf-4d91-be3c-acd2b980925e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:14.404007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1070/2148] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:16:14 2022] POST /identity/v3/projects => generated 378 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:14.408319 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f72a5-1c7a-4a5c-905c-b3968cebc0db admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:14.425050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f72a5-1c7a-4a5c-905c-b3968cebc0db admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:14.498751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43de612-3089-45ed-a6bd-8ea5ce938f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:14.499308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43de612-3089-45ed-a6bd-8ea5ce938f97 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.499643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43de612-3089-45ed-a6bd-8ea5ce938f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.500079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43de612-3089-45ed-a6bd-8ea5ce938f97 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.500828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43de612-3089-45ed-a6bd-8ea5ce938f97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:14.502951 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43de612-3089-45ed-a6bd-8ea5ce938f97 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:14.548062 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1161ddb-57bf-4d91-be3c-acd2b980925e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:14.551638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1079/2149] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:16:14 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:14.568912 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1161ddb-57bf-4d91-be3c-acd2b980925e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:14.584390 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4106537-a396-4d02-bd53-4c4ba57dc9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:14.585302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4106537-a396-4d02-bd53-4c4ba57dc9be admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.585698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4106537-a396-4d02-bd53-4c4ba57dc9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.586080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4106537-a396-4d02-bd53-4c4ba57dc9be admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.586943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4106537-a396-4d02-bd53-4c4ba57dc9be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:14.591169 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4106537-a396-4d02-bd53-4c4ba57dc9be admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:14.609628 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1071/2150] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:14 2022] GET /identity/v3/roles => generated 1427 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:16:14.613300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43de612-3089-45ed-a6bd-8ea5ce938f97 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:14.629770 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43de612-3089-45ed-a6bd-8ea5ce938f97 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:14.638003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8841d1-c68a-4828-9a18-7abc3d670184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:14.638980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8841d1-c68a-4828-9a18-7abc3d670184 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.639096 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8841d1-c68a-4828-9a18-7abc3d670184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.639495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8841d1-c68a-4828-9a18-7abc3d670184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.640138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8841d1-c68a-4828-9a18-7abc3d670184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:14.642465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8841d1-c68a-4828-9a18-7abc3d670184 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:14.660837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1080/2151] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:14 2022] GET /identity/v3/roles => generated 1427 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:14.661558 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4106537-a396-4d02-bd53-4c4ba57dc9be admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:14.673992 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4106537-a396-4d02-bd53-4c4ba57dc9be admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:14.679893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a3819-dc8f-4495-8e00-8e1789db1fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:14.681128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a3819-dc8f-4495-8e00-8e1789db1fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.681128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a3819-dc8f-4495-8e00-8e1789db1fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.681478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a3819-dc8f-4495-8e00-8e1789db1fc3 admin admin] PATH_INFO: `/v3/projects/e3439233c0b345a69ddbbf2a71fc7873/users/db78db02fe874a3db4b42cb9f421cfda/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.700776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a3819-dc8f-4495-8e00-8e1789db1fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3439233c0b345a69ddbbf2a71fc7873, user_id=db78db02fe874a3db4b42cb9f421cfda, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:14.704445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a3819-dc8f-4495-8e00-8e1789db1fc3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:14.726757 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1072/2152] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:14 2022] PUT /identity/v3/projects/e3439233c0b345a69ddbbf2a71fc7873/users/db78db02fe874a3db4b42cb9f421cfda/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:16:14.733612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36380b5-2c37-4a8c-8d22-d64d4f16d6db None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.734851 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36380b5-2c37-4a8c-8d22-d64d4f16d6db None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.735747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36380b5-2c37-4a8c-8d22-d64d4f16d6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.779002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f36380b5-2c37-4a8c-8d22-d64d4f16d6db None None] MFA Rules not processed for user `db78db02fe874a3db4b42cb9f421cfda`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:14.779840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f36380b5-2c37-4a8c-8d22-d64d4f16d6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:14.880021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1081/2153] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:14.886746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a3819-dc8f-4495-8e00-8e1789db1fc3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:14.929522 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:14.961440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a3819-dc8f-4495-8e00-8e1789db1fc3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:14.966211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27aff14-bba7-4fbe-9b0b-3a26fdb012fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:14.966865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27aff14-bba7-4fbe-9b0b-3a26fdb012fb admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:14.967152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27aff14-bba7-4fbe-9b0b-3a26fdb012fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:14.967430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27aff14-bba7-4fbe-9b0b-3a26fdb012fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:14.968168 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27aff14-bba7-4fbe-9b0b-3a26fdb012fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:14.970148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27aff14-bba7-4fbe-9b0b-3a26fdb012fb admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:14.999240 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1073/2154] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:16:14 2022] POST /identity/v3/projects => generated 378 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:15.003399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36380b5-2c37-4a8c-8d22-d64d4f16d6db None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.015034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36380b5-2c37-4a8c-8d22-d64d4f16d6db None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.055852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:15.092033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cef2a1-12bc-4d54-a071-6c499447e62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.092767 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cef2a1-12bc-4d54-a071-6c499447e62e admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.092953 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cef2a1-12bc-4d54-a071-6c499447e62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.093286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cef2a1-12bc-4d54-a071-6c499447e62e admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.094100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cef2a1-12bc-4d54-a071-6c499447e62e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.096763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cef2a1-12bc-4d54-a071-6c499447e62e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.123709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1082/2155] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:16:15 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:15.125048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27aff14-bba7-4fbe-9b0b-3a26fdb012fb admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.136597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27aff14-bba7-4fbe-9b0b-3a26fdb012fb admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.142111 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606a8a9-c577-44be-81b5-2fb72bebbd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.142916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606a8a9-c577-44be-81b5-2fb72bebbd15 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.143259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606a8a9-c577-44be-81b5-2fb72bebbd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.143609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606a8a9-c577-44be-81b5-2fb72bebbd15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.144456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606a8a9-c577-44be-81b5-2fb72bebbd15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.146368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606a8a9-c577-44be-81b5-2fb72bebbd15 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.159313 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1074/2156] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:15 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:15.163957 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cef2a1-12bc-4d54-a071-6c499447e62e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.174501 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cef2a1-12bc-4d54-a071-6c499447e62e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.179671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651be43-c9aa-4fc8-9d1b-e5ba1a3eac72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.180531 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651be43-c9aa-4fc8-9d1b-e5ba1a3eac72 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.180965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651be43-c9aa-4fc8-9d1b-e5ba1a3eac72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.181377 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651be43-c9aa-4fc8-9d1b-e5ba1a3eac72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.182234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651be43-c9aa-4fc8-9d1b-e5ba1a3eac72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.184230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651be43-c9aa-4fc8-9d1b-e5ba1a3eac72 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.200556 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1083/2157] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:15 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:15.202183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606a8a9-c577-44be-81b5-2fb72bebbd15 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.213833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606a8a9-c577-44be-81b5-2fb72bebbd15 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.219289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02b7ba-c520-432e-a5ad-52d2c948333a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.219979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02b7ba-c520-432e-a5ad-52d2c948333a admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.220265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02b7ba-c520-432e-a5ad-52d2c948333a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.220542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02b7ba-c520-432e-a5ad-52d2c948333a admin admin] PATH_INFO: `/v3/projects/33675a6052074acda66b47ad46088c48/users/8ef57d2d826a4c64b72305d56cc42a18/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.241263 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02b7ba-c520-432e-a5ad-52d2c948333a admin admin] RBAC: Authorizing `identity:create_grant(project_id=33675a6052074acda66b47ad46088c48, user_id=8ef57d2d826a4c64b72305d56cc42a18, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.244034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02b7ba-c520-432e-a5ad-52d2c948333a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.258812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1075/2158] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:15 2022] PUT /identity/v3/projects/33675a6052074acda66b47ad46088c48/users/8ef57d2d826a4c64b72305d56cc42a18/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:16:15.269957 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a44900-5454-4115-b180-0f04632f37b9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.270252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a44900-5454-4115-b180-0f04632f37b9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.270534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a44900-5454-4115-b180-0f04632f37b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.314214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-86a44900-5454-4115-b180-0f04632f37b9 None None] MFA Rules not processed for user `8ef57d2d826a4c64b72305d56cc42a18`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:15.316146 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-86a44900-5454-4115-b180-0f04632f37b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:15.414741 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1084/2159] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:15.416389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02b7ba-c520-432e-a5ad-52d2c948333a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.489111 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02b7ba-c520-432e-a5ad-52d2c948333a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.494616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f33c78-a7e5-4d0e-97d8-1b9233196326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.495516 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f33c78-a7e5-4d0e-97d8-1b9233196326 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.495805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f33c78-a7e5-4d0e-97d8-1b9233196326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.496162 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f33c78-a7e5-4d0e-97d8-1b9233196326 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.496858 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f33c78-a7e5-4d0e-97d8-1b9233196326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.499010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f33c78-a7e5-4d0e-97d8-1b9233196326 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.533485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1076/2160] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:16:15 2022] POST /identity/v3/projects => generated 378 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:16:15.540928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a44900-5454-4115-b180-0f04632f37b9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.557413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a44900-5454-4115-b180-0f04632f37b9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.649292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90421f5-58b1-4196-98f6-171dd6bf57b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.649292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90421f5-58b1-4196-98f6-171dd6bf57b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.649573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90421f5-58b1-4196-98f6-171dd6bf57b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.649871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90421f5-58b1-4196-98f6-171dd6bf57b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.650956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90421f5-58b1-4196-98f6-171dd6bf57b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.652907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90421f5-58b1-4196-98f6-171dd6bf57b9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.677826 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1085/2161] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:16:15 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:15.687055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f33c78-a7e5-4d0e-97d8-1b9233196326 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.698607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f33c78-a7e5-4d0e-97d8-1b9233196326 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.703776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27281cb-d1fb-444a-a7ba-43ec5decfef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.704413 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27281cb-d1fb-444a-a7ba-43ec5decfef6 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.704697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27281cb-d1fb-444a-a7ba-43ec5decfef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.704971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27281cb-d1fb-444a-a7ba-43ec5decfef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.705712 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27281cb-d1fb-444a-a7ba-43ec5decfef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.707687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27281cb-d1fb-444a-a7ba-43ec5decfef6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.719301 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1077/2162] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:15 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:15.724410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90421f5-58b1-4196-98f6-171dd6bf57b9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.735665 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90421f5-58b1-4196-98f6-171dd6bf57b9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.740426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ac5b8-347b-425c-9ba0-f1d4adbe866c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.741665 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ac5b8-347b-425c-9ba0-f1d4adbe866c admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.741945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ac5b8-347b-425c-9ba0-f1d4adbe866c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.742252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ac5b8-347b-425c-9ba0-f1d4adbe866c admin admin] PATH_INFO: `/v3/projects/7863ef9267014fa7970abebbbeefcbda/users/11263f19197848269fa72f54b87a4d48/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.758393 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ac5b8-347b-425c-9ba0-f1d4adbe866c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7863ef9267014fa7970abebbbeefcbda, user_id=11263f19197848269fa72f54b87a4d48, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.761112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ac5b8-347b-425c-9ba0-f1d4adbe866c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.776414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1086/2163] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:15 2022] PUT /identity/v3/projects/7863ef9267014fa7970abebbbeefcbda/users/11263f19197848269fa72f54b87a4d48/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:15.780402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27281cb-d1fb-444a-a7ba-43ec5decfef6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.802631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27281cb-d1fb-444a-a7ba-43ec5decfef6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.860547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:15.872544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afc05c-a512-4560-a170-45c5f21bad2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.873367 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afc05c-a512-4560-a170-45c5f21bad2e admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.873877 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afc05c-a512-4560-a170-45c5f21bad2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.874248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afc05c-a512-4560-a170-45c5f21bad2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.875090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afc05c-a512-4560-a170-45c5f21bad2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.876862 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afc05c-a512-4560-a170-45c5f21bad2e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.890098 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1078/2164] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:15 2022] GET /identity/v3/roles => generated 1427 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:15.894743 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ac5b8-347b-425c-9ba0-f1d4adbe866c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:15.910384 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ac5b8-347b-425c-9ba0-f1d4adbe866c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:15.917532 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647830a3-f89d-4163-893f-a54cae9a9b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:15.918613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647830a3-f89d-4163-893f-a54cae9a9b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.919085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647830a3-f89d-4163-893f-a54cae9a9b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.919518 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647830a3-f89d-4163-893f-a54cae9a9b32 admin admin] PATH_INFO: `/v3/projects/7863ef9267014fa7970abebbbeefcbda/users/11263f19197848269fa72f54b87a4d48/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:15.922243 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647830a3-f89d-4163-893f-a54cae9a9b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7863ef9267014fa7970abebbbeefcbda, user_id=11263f19197848269fa72f54b87a4d48, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:15.925148 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647830a3-f89d-4163-893f-a54cae9a9b32 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:15.936611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:15.950585 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1087/2165] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:15 2022] PUT /identity/v3/projects/7863ef9267014fa7970abebbbeefcbda/users/11263f19197848269fa72f54b87a4d48/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:15.962005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0c304-51d0-4cec-b8cd-c97e573f8146 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:15.963010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0c304-51d0-4cec-b8cd-c97e573f8146 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:15.963553 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0c304-51d0-4cec-b8cd-c97e573f8146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:16.004120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-43b0c304-51d0-4cec-b8cd-c97e573f8146 None None] MFA Rules not processed for user `11263f19197848269fa72f54b87a4d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:16.005414 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-43b0c304-51d0-4cec-b8cd-c97e573f8146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:16.121457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1079/2166] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:16.132873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647830a3-f89d-4163-893f-a54cae9a9b32 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:16.134603 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-647830a3-f89d-4163-893f-a54cae9a9b32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:16.203121 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647830a3-f89d-4163-893f-a54cae9a9b32 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:16.211277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4119cf-ca62-49dc-9f8b-ad0e4cb2c25c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:16.211973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4119cf-ca62-49dc-9f8b-ad0e4cb2c25c service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:16.212267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4119cf-ca62-49dc-9f8b-ad0e4cb2c25c service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:16.212548 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4119cf-ca62-49dc-9f8b-ad0e4cb2c25c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:16.222754 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4119cf-ca62-49dc-9f8b-ad0e4cb2c25c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:16.224214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4119cf-ca62-49dc-9f8b-ad0e4cb2c25c service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:16.237946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1088/2167] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:16.296882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0c304-51d0-4cec-b8cd-c97e573f8146 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:16.298201 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-43b0c304-51d0-4cec-b8cd-c97e573f8146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:16.311769 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0c304-51d0-4cec-b8cd-c97e573f8146 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:16.315613 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913482d6-ed19-4242-a067-4e2db168089b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:16.316244 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913482d6-ed19-4242-a067-4e2db168089b service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:16.316526 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913482d6-ed19-4242-a067-4e2db168089b service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:16.316805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913482d6-ed19-4242-a067-4e2db168089b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:16.391125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913482d6-ed19-4242-a067-4e2db168089b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:16.392618 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913482d6-ed19-4242-a067-4e2db168089b service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:16.403749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1080/2168] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:16:19.445168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4119cf-ca62-49dc-9f8b-ad0e4cb2c25c service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:19.549667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4119cf-ca62-49dc-9f8b-ad0e4cb2c25c service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:19.554272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96812dd2-6008-450b-b415-ba202c078d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:19.554975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96812dd2-6008-450b-b415-ba202c078d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:19.555259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96812dd2-6008-450b-b415-ba202c078d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:19.555533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96812dd2-6008-450b-b415-ba202c078d82 admin admin] PATH_INFO: `/v3/users/db78db02fe874a3db4b42cb9f421cfda` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:19.556900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96812dd2-6008-450b-b415-ba202c078d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db78db02fe874a3db4b42cb9f421cfda)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:19.558901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96812dd2-6008-450b-b415-ba202c078d82 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:19.620447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96812dd2-6008-450b-b415-ba202c078d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db78db02fe874a3db4b42cb9f421cfda'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:19.622093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96812dd2-6008-450b-b415-ba202c078d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db78db02fe874a3db4b42cb9f421cfda'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:19.632236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96812dd2-6008-450b-b415-ba202c078d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db78db02fe874a3db4b42cb9f421cfda'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:19.633050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96812dd2-6008-450b-b415-ba202c078d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db78db02fe874a3db4b42cb9f421cfda'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:19.645403 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-96812dd2-6008-450b-b415-ba202c078d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db78db02fe874a3db4b42cb9f421cfda'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:19.660048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1089/2169] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:16:19 2022] DELETE /identity/v3/users/db78db02fe874a3db4b42cb9f421cfda => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:19.664666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddaef2-d49d-4c75-85cf-4477bc1e82b9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:19.665154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddaef2-d49d-4c75-85cf-4477bc1e82b9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:19.665447 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddaef2-d49d-4c75-85cf-4477bc1e82b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:19.690343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-58ddaef2-d49d-4c75-85cf-4477bc1e82b9 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:19.691531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-58ddaef2-d49d-4c75-85cf-4477bc1e82b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:19.758106 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1081/2170] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:19.767983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96812dd2-6008-450b-b415-ba202c078d82 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:19.769428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-96812dd2-6008-450b-b415-ba202c078d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:19.802812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:19.840176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96812dd2-6008-450b-b415-ba202c078d82 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:19.844473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6ece8-423e-46d6-8d04-a319bc475877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:19.845109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6ece8-423e-46d6-8d04-a319bc475877 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:19.845396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6ece8-423e-46d6-8d04-a319bc475877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:19.845707 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6ece8-423e-46d6-8d04-a319bc475877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:19.864366 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6ece8-423e-46d6-8d04-a319bc475877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:19.867091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6ece8-423e-46d6-8d04-a319bc475877 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:19.894619 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1090/2171] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:20.191141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddaef2-d49d-4c75-85cf-4477bc1e82b9 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:20.192916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-58ddaef2-d49d-4c75-85cf-4477bc1e82b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:20.207199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddaef2-d49d-4c75-85cf-4477bc1e82b9 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:20.212766 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:20.213556 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:20.214445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:20.214445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] PATH_INFO: `/v3/projects/e3439233c0b345a69ddbbf2a71fc7873` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:20.215778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3439233c0b345a69ddbbf2a71fc7873)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:20.218367 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:20.231451 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:20.284085 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3439233c0b345a69ddbbf2a71fc7873', 'request_id': 'req-d0692976-41b5-4e54-90de-5f8b118e9114'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:20.304857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3439233c0b345a69ddbbf2a71fc7873', 'request_id': 'req-d0692976-41b5-4e54-90de-5f8b118e9114'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:20.318382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3439233c0b345a69ddbbf2a71fc7873', 'request_id': 'req-d0692976-41b5-4e54-90de-5f8b118e9114'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:20.334329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] The token cache is being invalidate because project e3439233c0b345a69ddbbf2a71fc7873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:20.335452 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:20.336580 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:20.338640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1082/2172] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:16:20 2022] DELETE /identity/v3/projects/e3439233c0b345a69ddbbf2a71fc7873 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:16:20.342648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6ece8-423e-46d6-8d04-a319bc475877 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:20.354322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6ece8-423e-46d6-8d04-a319bc475877 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:20.451123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26eb24-8a08-4a61-adcd-ba3f64e2b129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:20.451896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26eb24-8a08-4a61-adcd-ba3f64e2b129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:20.452069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26eb24-8a08-4a61-adcd-ba3f64e2b129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:20.452348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26eb24-8a08-4a61-adcd-ba3f64e2b129 admin admin] PATH_INFO: `/v3/users/8ef57d2d826a4c64b72305d56cc42a18` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:20.454576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26eb24-8a08-4a61-adcd-ba3f64e2b129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef57d2d826a4c64b72305d56cc42a18)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:20.457382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da26eb24-8a08-4a61-adcd-ba3f64e2b129 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:20.524606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da26eb24-8a08-4a61-adcd-ba3f64e2b129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef57d2d826a4c64b72305d56cc42a18'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:20.529381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da26eb24-8a08-4a61-adcd-ba3f64e2b129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef57d2d826a4c64b72305d56cc42a18'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:20.534510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da26eb24-8a08-4a61-adcd-ba3f64e2b129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef57d2d826a4c64b72305d56cc42a18'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:20.535245 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da26eb24-8a08-4a61-adcd-ba3f64e2b129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef57d2d826a4c64b72305d56cc42a18'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:20.554199 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-da26eb24-8a08-4a61-adcd-ba3f64e2b129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef57d2d826a4c64b72305d56cc42a18'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:20.568654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1091/2173] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:16:20 2022] DELETE /identity/v3/users/8ef57d2d826a4c64b72305d56cc42a18 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:20.860726 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:20.863146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:20.880343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0692976-41b5-4e54-90de-5f8b118e9114 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:20.998522 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:20.999265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:20.999620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:20.999972 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] PATH_INFO: `/v3/projects/33675a6052074acda66b47ad46088c48` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:21.001322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33675a6052074acda66b47ad46088c48)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:21.003436 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:21.093351 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33675a6052074acda66b47ad46088c48', 'request_id': 'req-21a324fb-820a-48b5-b9a9-01b34b379fd9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:21.108963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33675a6052074acda66b47ad46088c48', 'request_id': 'req-21a324fb-820a-48b5-b9a9-01b34b379fd9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:21.133631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33675a6052074acda66b47ad46088c48', 'request_id': 'req-21a324fb-820a-48b5-b9a9-01b34b379fd9'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:21.165749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] The token cache is being invalidate because project 33675a6052074acda66b47ad46088c48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:21.166177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:21.167363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:21.169654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1083/2174] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:16:20 2022] DELETE /identity/v3/projects/33675a6052074acda66b47ad46088c48 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:21.173613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26eb24-8a08-4a61-adcd-ba3f64e2b129 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:21.175549 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-da26eb24-8a08-4a61-adcd-ba3f64e2b129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:21.192814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26eb24-8a08-4a61-adcd-ba3f64e2b129 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:21.244610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:21.269625 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad873aa-0e1a-4aef-86bd-7c64a753d231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:21.270297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad873aa-0e1a-4aef-86bd-7c64a753d231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:21.270599 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad873aa-0e1a-4aef-86bd-7c64a753d231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:21.270907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad873aa-0e1a-4aef-86bd-7c64a753d231 admin admin] PATH_INFO: `/v3/users/11263f19197848269fa72f54b87a4d48` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:21.272166 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad873aa-0e1a-4aef-86bd-7c64a753d231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11263f19197848269fa72f54b87a4d48)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:21.274095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad873aa-0e1a-4aef-86bd-7c64a753d231 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:21.338371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cad873aa-0e1a-4aef-86bd-7c64a753d231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11263f19197848269fa72f54b87a4d48'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:21.339421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cad873aa-0e1a-4aef-86bd-7c64a753d231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11263f19197848269fa72f54b87a4d48'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:21.348672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cad873aa-0e1a-4aef-86bd-7c64a753d231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11263f19197848269fa72f54b87a4d48'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:21.352269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cad873aa-0e1a-4aef-86bd-7c64a753d231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11263f19197848269fa72f54b87a4d48'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:21.370015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cad873aa-0e1a-4aef-86bd-7c64a753d231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11263f19197848269fa72f54b87a4d48'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:21.395040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1092/2175] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:16:21 2022] DELETE /identity/v3/users/11263f19197848269fa72f54b87a4d48 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:22.083629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:22.087910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:22.114453 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a324fb-820a-48b5-b9a9-01b34b379fd9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:22.140857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:22.180795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c483a719-c901-48f1-966a-3e09c1d3deba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:22.182871 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483a719-c901-48f1-966a-3e09c1d3deba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:22.183580 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483a719-c901-48f1-966a-3e09c1d3deba admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:22.184001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c483a719-c901-48f1-966a-3e09c1d3deba admin admin] PATH_INFO: `/v3/projects/7863ef9267014fa7970abebbbeefcbda` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:22.187440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483a719-c901-48f1-966a-3e09c1d3deba admin admin] RBAC: Authorizing `identity:delete_project(project_id=7863ef9267014fa7970abebbbeefcbda)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:22.189472 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c483a719-c901-48f1-966a-3e09c1d3deba admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:22.251661 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c483a719-c901-48f1-966a-3e09c1d3deba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7863ef9267014fa7970abebbbeefcbda', 'request_id': 'req-c483a719-c901-48f1-966a-3e09c1d3deba'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:22.259348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c483a719-c901-48f1-966a-3e09c1d3deba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7863ef9267014fa7970abebbbeefcbda', 'request_id': 'req-c483a719-c901-48f1-966a-3e09c1d3deba'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:22.268329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c483a719-c901-48f1-966a-3e09c1d3deba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7863ef9267014fa7970abebbbeefcbda', 'request_id': 'req-c483a719-c901-48f1-966a-3e09c1d3deba'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:22.281342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c483a719-c901-48f1-966a-3e09c1d3deba admin admin] The token cache is being invalidate because project 7863ef9267014fa7970abebbbeefcbda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:22.281640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c483a719-c901-48f1-966a-3e09c1d3deba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:22.282510 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c483a719-c901-48f1-966a-3e09c1d3deba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:22.284199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1084/2176] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:16:22 2022] DELETE /identity/v3/projects/7863ef9267014fa7970abebbbeefcbda => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:22.290072 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42588832-f49b-43f1-b51b-9008742d94ea None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:22.290557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42588832-f49b-43f1-b51b-9008742d94ea None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:22.291181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42588832-f49b-43f1-b51b-9008742d94ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:22.313372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-42588832-f49b-43f1-b51b-9008742d94ea None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:22.314739 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-42588832-f49b-43f1-b51b-9008742d94ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:22.390849 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1093/2177] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:22.393557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a26c6a-99f6-404c-8a8e-929b17e82e57 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:22.393789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a26c6a-99f6-404c-8a8e-929b17e82e57 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:22.394077 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a26c6a-99f6-404c-8a8e-929b17e82e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:22.419493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d4a26c6a-99f6-404c-8a8e-929b17e82e57 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:22.420783 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d4a26c6a-99f6-404c-8a8e-929b17e82e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:22.430813 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1085/2178] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:22.434240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42588832-f49b-43f1-b51b-9008742d94ea None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:22.446166 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42588832-f49b-43f1-b51b-9008742d94ea None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:22.452175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc491e-67e9-4a71-9189-3fd095748974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:22.453101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc491e-67e9-4a71-9189-3fd095748974 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:22.453568 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc491e-67e9-4a71-9189-3fd095748974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:22.454041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc491e-67e9-4a71-9189-3fd095748974 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:22.454964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc491e-67e9-4a71-9189-3fd095748974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:16:22.455452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc491e-67e9-4a71-9189-3fd095748974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:22.457581 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc491e-67e9-4a71-9189-3fd095748974 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:22.472758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1094/2179] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:16:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:16:22.477542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a26c6a-99f6-404c-8a8e-929b17e82e57 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:22.488305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a26c6a-99f6-404c-8a8e-929b17e82e57 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:22.493966 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ae0f7-95c5-42f0-8ea8-f80eb5b4e25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:22.494606 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ae0f7-95c5-42f0-8ea8-f80eb5b4e25d admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:22.495440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ae0f7-95c5-42f0-8ea8-f80eb5b4e25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:22.495440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ae0f7-95c5-42f0-8ea8-f80eb5b4e25d admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:22.495942 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ae0f7-95c5-42f0-8ea8-f80eb5b4e25d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:22.497925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ae0f7-95c5-42f0-8ea8-f80eb5b4e25d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:22.555750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1086/2180] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:16:22 2022] POST /identity/v3/projects => generated 384 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:22.557718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc491e-67e9-4a71-9189-3fd095748974 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:22.572566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc491e-67e9-4a71-9189-3fd095748974 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:22.621397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:22.635887 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47944d-6672-4e13-ba96-c5ae360f6e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:22.636513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47944d-6672-4e13-ba96-c5ae360f6e18 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:22.636845 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47944d-6672-4e13-ba96-c5ae360f6e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:22.637125 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47944d-6672-4e13-ba96-c5ae360f6e18 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:22.637802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47944d-6672-4e13-ba96-c5ae360f6e18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:22.639843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47944d-6672-4e13-ba96-c5ae360f6e18 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:22.667252 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1095/2181] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:16:22 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:22.668782 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ae0f7-95c5-42f0-8ea8-f80eb5b4e25d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:22.680262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ae0f7-95c5-42f0-8ea8-f80eb5b4e25d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:22.685399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffaabc-20fa-4f4d-9677-6c3b5034ac20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:22.686023 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffaabc-20fa-4f4d-9677-6c3b5034ac20 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:22.686307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffaabc-20fa-4f4d-9677-6c3b5034ac20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:22.686700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffaabc-20fa-4f4d-9677-6c3b5034ac20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:22.687314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ffaabc-20fa-4f4d-9677-6c3b5034ac20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:22.688951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ffaabc-20fa-4f4d-9677-6c3b5034ac20 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:22.700744 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1087/2182] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:22 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:22.704748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47944d-6672-4e13-ba96-c5ae360f6e18 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:22.715394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47944d-6672-4e13-ba96-c5ae360f6e18 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:22.720151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c40087d-5b34-4079-b50f-1d199c98aa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:22.720763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40087d-5b34-4079-b50f-1d199c98aa33 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:22.721049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40087d-5b34-4079-b50f-1d199c98aa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:22.721328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c40087d-5b34-4079-b50f-1d199c98aa33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:22.722112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40087d-5b34-4079-b50f-1d199c98aa33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:22.723774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c40087d-5b34-4079-b50f-1d199c98aa33 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:22.736118 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1096/2183] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:22 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:22.739115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffaabc-20fa-4f4d-9677-6c3b5034ac20 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:22.750150 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffaabc-20fa-4f4d-9677-6c3b5034ac20 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:22.755009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52426c37-81a1-44d7-af80-9672d1a9d730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:22.755752 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52426c37-81a1-44d7-af80-9672d1a9d730 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:22.756138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52426c37-81a1-44d7-af80-9672d1a9d730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:22.757042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52426c37-81a1-44d7-af80-9672d1a9d730 admin admin] PATH_INFO: `/v3/projects/603d9be497744e84be2baaad518f1af2/users/7ceab47f5375435380f73d14d5c24484/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:22.777856 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52426c37-81a1-44d7-af80-9672d1a9d730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=603d9be497744e84be2baaad518f1af2, user_id=7ceab47f5375435380f73d14d5c24484, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:22.780906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52426c37-81a1-44d7-af80-9672d1a9d730 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:22.802789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1088/2184] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:22 2022] PUT /identity/v3/projects/603d9be497744e84be2baaad518f1af2/users/7ceab47f5375435380f73d14d5c24484/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:22.815352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02554b16-535d-450c-8987-acddf2d28db1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:22.818138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02554b16-535d-450c-8987-acddf2d28db1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:22.818138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02554b16-535d-450c-8987-acddf2d28db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:22.848984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-02554b16-535d-450c-8987-acddf2d28db1 None None] MFA Rules not processed for user `7ceab47f5375435380f73d14d5c24484`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:22.855168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-02554b16-535d-450c-8987-acddf2d28db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:22.954690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1097/2185] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:22.954901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52426c37-81a1-44d7-af80-9672d1a9d730 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:23.015402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:23.021352 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52426c37-81a1-44d7-af80-9672d1a9d730 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:23.029122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed0e60-46ba-4896-9aee-75bc99bf2b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:23.029889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed0e60-46ba-4896-9aee-75bc99bf2b43 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:23.030444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed0e60-46ba-4896-9aee-75bc99bf2b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:23.031281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed0e60-46ba-4896-9aee-75bc99bf2b43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:23.032270 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ed0e60-46ba-4896-9aee-75bc99bf2b43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:23.035140 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ed0e60-46ba-4896-9aee-75bc99bf2b43 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:23.066775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1089/2186] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:16:22 2022] POST /identity/v3/projects => generated 384 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:23.070648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02554b16-535d-450c-8987-acddf2d28db1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:23.082449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02554b16-535d-450c-8987-acddf2d28db1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:23.147254 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44ba10-545a-4393-8645-379b55f4a751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:23.147880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44ba10-545a-4393-8645-379b55f4a751 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:23.148156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44ba10-545a-4393-8645-379b55f4a751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:23.148427 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db44ba10-545a-4393-8645-379b55f4a751 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:23.149259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db44ba10-545a-4393-8645-379b55f4a751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:23.151699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db44ba10-545a-4393-8645-379b55f4a751 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:23.177648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1098/2187] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:16:23 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:23.182338 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed0e60-46ba-4896-9aee-75bc99bf2b43 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:23.193104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed0e60-46ba-4896-9aee-75bc99bf2b43 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:23.198116 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bb7ab-44a6-4992-ba95-df4c90a3583f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:23.198833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bb7ab-44a6-4992-ba95-df4c90a3583f admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:23.199280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bb7ab-44a6-4992-ba95-df4c90a3583f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:23.199641 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bb7ab-44a6-4992-ba95-df4c90a3583f admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:23.200130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bb7ab-44a6-4992-ba95-df4c90a3583f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:23.201843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bb7ab-44a6-4992-ba95-df4c90a3583f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:23.214137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1090/2188] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:23 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:23.218292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44ba10-545a-4393-8645-379b55f4a751 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:23.230216 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db44ba10-545a-4393-8645-379b55f4a751 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:23.235509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156605fd-6a68-44f3-811b-b38964e8dafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:23.236274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156605fd-6a68-44f3-811b-b38964e8dafe admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:23.236646 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156605fd-6a68-44f3-811b-b38964e8dafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:23.236993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156605fd-6a68-44f3-811b-b38964e8dafe admin admin] PATH_INFO: `/v3/projects/da14d8e0f08448e4a58b3372af690e73/users/b07bb341258f4ad193a12819b6ba3ca0/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:23.253618 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156605fd-6a68-44f3-811b-b38964e8dafe admin admin] RBAC: Authorizing `identity:create_grant(project_id=da14d8e0f08448e4a58b3372af690e73, user_id=b07bb341258f4ad193a12819b6ba3ca0, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:23.256483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156605fd-6a68-44f3-811b-b38964e8dafe admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:23.270929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1099/2189] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:23 2022] PUT /identity/v3/projects/da14d8e0f08448e4a58b3372af690e73/users/b07bb341258f4ad193a12819b6ba3ca0/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:23.275858 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bb7ab-44a6-4992-ba95-df4c90a3583f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:23.288289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bb7ab-44a6-4992-ba95-df4c90a3583f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:23.347634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b32415-aca3-4a10-b371-18c94a74ac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:23.348254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b32415-aca3-4a10-b371-18c94a74ac69 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:23.348533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b32415-aca3-4a10-b371-18c94a74ac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:23.348897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b32415-aca3-4a10-b371-18c94a74ac69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:23.349597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b32415-aca3-4a10-b371-18c94a74ac69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:23.351805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b32415-aca3-4a10-b371-18c94a74ac69 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:23.362040 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1091/2190] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:23 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:16:23.366370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156605fd-6a68-44f3-811b-b38964e8dafe admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:23.376614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156605fd-6a68-44f3-811b-b38964e8dafe admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:23.381362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232efc5-35d1-4e17-9dd9-7bdf50c3a101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:23.382058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232efc5-35d1-4e17-9dd9-7bdf50c3a101 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:23.382343 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232efc5-35d1-4e17-9dd9-7bdf50c3a101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:23.382620 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232efc5-35d1-4e17-9dd9-7bdf50c3a101 admin admin] PATH_INFO: `/v3/projects/da14d8e0f08448e4a58b3372af690e73/users/b07bb341258f4ad193a12819b6ba3ca0/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:23.384612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232efc5-35d1-4e17-9dd9-7bdf50c3a101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da14d8e0f08448e4a58b3372af690e73, user_id=b07bb341258f4ad193a12819b6ba3ca0, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:23.387292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232efc5-35d1-4e17-9dd9-7bdf50c3a101 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:23.398291 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1100/2191] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:23 2022] PUT /identity/v3/projects/da14d8e0f08448e4a58b3372af690e73/users/b07bb341258f4ad193a12819b6ba3ca0/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:23.408485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfc6a7-78f7-4408-bc54-f18bd917655b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:23.408775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfc6a7-78f7-4408-bc54-f18bd917655b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:23.409055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfc6a7-78f7-4408-bc54-f18bd917655b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:23.441194 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-70cfc6a7-78f7-4408-bc54-f18bd917655b None None] MFA Rules not processed for user `b07bb341258f4ad193a12819b6ba3ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:23.442296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-70cfc6a7-78f7-4408-bc54-f18bd917655b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:23.520041 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1092/2192] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:23.531986 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232efc5-35d1-4e17-9dd9-7bdf50c3a101 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:23.533914 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8232efc5-35d1-4e17-9dd9-7bdf50c3a101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:23.552032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:23.604989 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232efc5-35d1-4e17-9dd9-7bdf50c3a101 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:23.608803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d4e3e-c8d3-43b4-b440-60c52903549a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:23.609467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d4e3e-c8d3-43b4-b440-60c52903549a service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:23.609747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d4e3e-c8d3-43b4-b440-60c52903549a service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:23.610021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d4e3e-c8d3-43b4-b440-60c52903549a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:23.672576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d4e3e-c8d3-43b4-b440-60c52903549a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:23.674033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d4e3e-c8d3-43b4-b440-60c52903549a service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:23.684012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1101/2193] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:29.087133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfc6a7-78f7-4408-bc54-f18bd917655b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:29.096912 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfc6a7-78f7-4408-bc54-f18bd917655b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:29.100804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d92d95-93b2-4753-abab-648b98665df9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:29.101639 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d92d95-93b2-4753-abab-648b98665df9 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:29.101994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d92d95-93b2-4753-abab-648b98665df9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:29.102524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d92d95-93b2-4753-abab-648b98665df9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:29.112971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d92d95-93b2-4753-abab-648b98665df9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:29.114413 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d92d95-93b2-4753-abab-648b98665df9 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:29.124348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1093/2194] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:33.679539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=43 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:34.074868 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d4e3e-c8d3-43b4-b440-60c52903549a service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:34.156414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d4e3e-c8d3-43b4-b440-60c52903549a service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:34.161198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dd6c8-a113-45ba-95cd-49ade92dcd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:34.161848 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dd6c8-a113-45ba-95cd-49ade92dcd36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:34.162843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dd6c8-a113-45ba-95cd-49ade92dcd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:34.163258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dd6c8-a113-45ba-95cd-49ade92dcd36 admin admin] PATH_INFO: `/v3/users/7ceab47f5375435380f73d14d5c24484` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:34.168498 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68dd6c8-a113-45ba-95cd-49ade92dcd36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ceab47f5375435380f73d14d5c24484)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:34.170580 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68dd6c8-a113-45ba-95cd-49ade92dcd36 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:34.223557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e68dd6c8-a113-45ba-95cd-49ade92dcd36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ceab47f5375435380f73d14d5c24484'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:34.224586 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e68dd6c8-a113-45ba-95cd-49ade92dcd36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ceab47f5375435380f73d14d5c24484'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:34.236749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e68dd6c8-a113-45ba-95cd-49ade92dcd36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ceab47f5375435380f73d14d5c24484'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:34.237814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e68dd6c8-a113-45ba-95cd-49ade92dcd36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ceab47f5375435380f73d14d5c24484'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:34.263759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e68dd6c8-a113-45ba-95cd-49ade92dcd36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ceab47f5375435380f73d14d5c24484'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:34.281124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1102/2195] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:16:34 2022] DELETE /identity/v3/users/7ceab47f5375435380f73d14d5c24484 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:34.285645 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f480b-6f04-487f-bfbf-d76279d453ea None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:34.286455 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f480b-6f04-487f-bfbf-d76279d453ea None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:34.286455 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f480b-6f04-487f-bfbf-d76279d453ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:34.309340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a88f480b-6f04-487f-bfbf-d76279d453ea None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:34.310566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a88f480b-6f04-487f-bfbf-d76279d453ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:34.352991 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:34.389374 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1094/2196] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:16:34.399527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dd6c8-a113-45ba-95cd-49ade92dcd36 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:34.401015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e68dd6c8-a113-45ba-95cd-49ade92dcd36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:34.466307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dd6c8-a113-45ba-95cd-49ade92dcd36 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:34.470105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eac011-56c8-4037-9a23-a22dd8bb1ff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:34.470782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eac011-56c8-4037-9a23-a22dd8bb1ff5 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:34.471067 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eac011-56c8-4037-9a23-a22dd8bb1ff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:34.471343 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eac011-56c8-4037-9a23-a22dd8bb1ff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:34.488695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eac011-56c8-4037-9a23-a22dd8bb1ff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:34.490828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eac011-56c8-4037-9a23-a22dd8bb1ff5 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:34.503175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1103/2197] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:34.805719 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f480b-6f04-487f-bfbf-d76279d453ea None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:34.808162 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a88f480b-6f04-487f-bfbf-d76279d453ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:34.828627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f480b-6f04-487f-bfbf-d76279d453ea None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:34.837575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:34.838336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:34.838709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:34.839061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] PATH_INFO: `/v3/projects/603d9be497744e84be2baaad518f1af2` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:34.840999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] RBAC: Authorizing `identity:delete_project(project_id=603d9be497744e84be2baaad518f1af2)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:34.843880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:34.906227 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '603d9be497744e84be2baaad518f1af2', 'request_id': 'req-69232e16-1e38-423d-a645-9bd3e05dbcea'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:34.916298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '603d9be497744e84be2baaad518f1af2', 'request_id': 'req-69232e16-1e38-423d-a645-9bd3e05dbcea'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:34.925691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '603d9be497744e84be2baaad518f1af2', 'request_id': 'req-69232e16-1e38-423d-a645-9bd3e05dbcea'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:34.937272 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] The token cache is being invalidate because project 603d9be497744e84be2baaad518f1af2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:34.937605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:34.938269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:34.939829 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1095/2198] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:16:34 2022] DELETE /identity/v3/projects/603d9be497744e84be2baaad518f1af2 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:34.944316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eac011-56c8-4037-9a23-a22dd8bb1ff5 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:34.954616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eac011-56c8-4037-9a23-a22dd8bb1ff5 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:34.974738 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:35.019307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2f664-f558-42cf-ab9a-94148df3bd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:35.019961 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2f664-f558-42cf-ab9a-94148df3bd64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:35.020246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2f664-f558-42cf-ab9a-94148df3bd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:35.020522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2f664-f558-42cf-ab9a-94148df3bd64 admin admin] PATH_INFO: `/v3/users/b07bb341258f4ad193a12819b6ba3ca0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:35.022270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2f664-f558-42cf-ab9a-94148df3bd64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b07bb341258f4ad193a12819b6ba3ca0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:35.024241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2f664-f558-42cf-ab9a-94148df3bd64 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:35.078837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-70c2f664-f558-42cf-ab9a-94148df3bd64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b07bb341258f4ad193a12819b6ba3ca0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:35.079829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-70c2f664-f558-42cf-ab9a-94148df3bd64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b07bb341258f4ad193a12819b6ba3ca0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:35.101230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-70c2f664-f558-42cf-ab9a-94148df3bd64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b07bb341258f4ad193a12819b6ba3ca0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:35.101230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-70c2f664-f558-42cf-ab9a-94148df3bd64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b07bb341258f4ad193a12819b6ba3ca0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:35.118837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-70c2f664-f558-42cf-ab9a-94148df3bd64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b07bb341258f4ad193a12819b6ba3ca0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:35.139346 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1104/2199] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:16:34 2022] DELETE /identity/v3/users/b07bb341258f4ad193a12819b6ba3ca0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:35.551547 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:35.553857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:35.584337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69232e16-1e38-423d-a645-9bd3e05dbcea admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:35.678128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52abd71-9af5-47e7-986e-220a6bbe6a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:35.678128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52abd71-9af5-47e7-986e-220a6bbe6a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:35.678448 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52abd71-9af5-47e7-986e-220a6bbe6a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:35.678569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52abd71-9af5-47e7-986e-220a6bbe6a97 admin admin] PATH_INFO: `/v3/projects/da14d8e0f08448e4a58b3372af690e73` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:35.679900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52abd71-9af5-47e7-986e-220a6bbe6a97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da14d8e0f08448e4a58b3372af690e73)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:35.682187 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52abd71-9af5-47e7-986e-220a6bbe6a97 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:35.747132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d52abd71-9af5-47e7-986e-220a6bbe6a97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da14d8e0f08448e4a58b3372af690e73', 'request_id': 'req-d52abd71-9af5-47e7-986e-220a6bbe6a97'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:35.753503 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d52abd71-9af5-47e7-986e-220a6bbe6a97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da14d8e0f08448e4a58b3372af690e73', 'request_id': 'req-d52abd71-9af5-47e7-986e-220a6bbe6a97'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:35.762849 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d52abd71-9af5-47e7-986e-220a6bbe6a97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da14d8e0f08448e4a58b3372af690e73', 'request_id': 'req-d52abd71-9af5-47e7-986e-220a6bbe6a97'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:35.766296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:35.780304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d52abd71-9af5-47e7-986e-220a6bbe6a97 admin admin] The token cache is being invalidate because project da14d8e0f08448e4a58b3372af690e73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:35.780723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d52abd71-9af5-47e7-986e-220a6bbe6a97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:35.781484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d52abd71-9af5-47e7-986e-220a6bbe6a97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:35.783094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1096/2200] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:16:35 2022] DELETE /identity/v3/projects/da14d8e0f08448e4a58b3372af690e73 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:35.789243 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b9e13-407a-4db2-9300-03047ec75fde None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:35.789537 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b9e13-407a-4db2-9300-03047ec75fde None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:35.789818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b9e13-407a-4db2-9300-03047ec75fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:35.812496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2a0b9e13-407a-4db2-9300-03047ec75fde None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:35.813753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2a0b9e13-407a-4db2-9300-03047ec75fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:35.887487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1105/2201] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:35.897195 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e8ebc-2ce5-43d0-9505-e26769090c72 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:35.897567 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e8ebc-2ce5-43d0-9505-e26769090c72 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:35.897890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e8ebc-2ce5-43d0-9505-e26769090c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:35.920264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-472e8ebc-2ce5-43d0-9505-e26769090c72 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:35.921529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-472e8ebc-2ce5-43d0-9505-e26769090c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:35.930181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1097/2202] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:35.934624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b9e13-407a-4db2-9300-03047ec75fde None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:35.948504 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b9e13-407a-4db2-9300-03047ec75fde None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:35.956877 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38accb-e54f-4fce-9647-3c5d252c14cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:35.957925 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38accb-e54f-4fce-9647-3c5d252c14cb admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:35.958426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38accb-e54f-4fce-9647-3c5d252c14cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:35.959505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38accb-e54f-4fce-9647-3c5d252c14cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:35.960745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38accb-e54f-4fce-9647-3c5d252c14cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:16:35.961258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38accb-e54f-4fce-9647-3c5d252c14cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:35.964179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38accb-e54f-4fce-9647-3c5d252c14cb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:35.984861 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1106/2203] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:16:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:16:35.989280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e8ebc-2ce5-43d0-9505-e26769090c72 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:36.009856 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472e8ebc-2ce5-43d0-9505-e26769090c72 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:36.015446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6241e0-f3bc-4436-abe0-5b67178b9bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:36.016077 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6241e0-f3bc-4436-abe0-5b67178b9bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:36.016362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6241e0-f3bc-4436-abe0-5b67178b9bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:36.016639 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6241e0-f3bc-4436-abe0-5b67178b9bb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:36.017327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6241e0-f3bc-4436-abe0-5b67178b9bb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:36.019310 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6241e0-f3bc-4436-abe0-5b67178b9bb2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:36.048690 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1098/2204] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:16:35 2022] POST /identity/v3/projects => generated 374 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:16:36.055315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38accb-e54f-4fce-9647-3c5d252c14cb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:36.069126 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38accb-e54f-4fce-9647-3c5d252c14cb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:36.099446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:36.145778 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb9ed2-d284-4e10-bdda-47e3897b8d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:36.146491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb9ed2-d284-4e10-bdda-47e3897b8d8f admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:36.146811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb9ed2-d284-4e10-bdda-47e3897b8d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:36.147091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb9ed2-d284-4e10-bdda-47e3897b8d8f admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:36.147773 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb9ed2-d284-4e10-bdda-47e3897b8d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:36.149719 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb9ed2-d284-4e10-bdda-47e3897b8d8f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:36.178788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1107/2205] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:16:36 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:36.187373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6241e0-f3bc-4436-abe0-5b67178b9bb2 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:36.211672 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6241e0-f3bc-4436-abe0-5b67178b9bb2 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:36.221034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf82e6d-2a19-4f1b-a46c-62278b3dc35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:36.222031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf82e6d-2a19-4f1b-a46c-62278b3dc35c admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:36.222477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf82e6d-2a19-4f1b-a46c-62278b3dc35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:36.222971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf82e6d-2a19-4f1b-a46c-62278b3dc35c admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:36.224146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf82e6d-2a19-4f1b-a46c-62278b3dc35c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:36.227033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf82e6d-2a19-4f1b-a46c-62278b3dc35c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:36.246723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1099/2206] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:36 2022] GET /identity/v3/roles => generated 1427 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:36.250103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb9ed2-d284-4e10-bdda-47e3897b8d8f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:36.274253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb9ed2-d284-4e10-bdda-47e3897b8d8f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:36.280076 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b4706-e1cc-41c4-b803-eb86db4358a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:36.280975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b4706-e1cc-41c4-b803-eb86db4358a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:36.281259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b4706-e1cc-41c4-b803-eb86db4358a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:36.281536 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b4706-e1cc-41c4-b803-eb86db4358a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:36.282256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b4706-e1cc-41c4-b803-eb86db4358a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:36.284479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899b4706-e1cc-41c4-b803-eb86db4358a8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:36.307928 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1108/2207] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:36 2022] GET /identity/v3/roles => generated 1427 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:36.313906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf82e6d-2a19-4f1b-a46c-62278b3dc35c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:36.326821 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf82e6d-2a19-4f1b-a46c-62278b3dc35c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:36.333417 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d516937-2a0b-4280-84cb-d5f4352545fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:36.334414 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516937-2a0b-4280-84cb-d5f4352545fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:36.334920 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516937-2a0b-4280-84cb-d5f4352545fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:36.335434 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516937-2a0b-4280-84cb-d5f4352545fc admin admin] PATH_INFO: `/v3/projects/9a171a0c8da74b03bc0b2450968005e7/users/0ab7eb2d1199454e83b48b9d1b0e879c/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:36.359393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d516937-2a0b-4280-84cb-d5f4352545fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a171a0c8da74b03bc0b2450968005e7, user_id=0ab7eb2d1199454e83b48b9d1b0e879c, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:36.363535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d516937-2a0b-4280-84cb-d5f4352545fc admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:36.390095 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1100/2208] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:36 2022] PUT /identity/v3/projects/9a171a0c8da74b03bc0b2450968005e7/users/0ab7eb2d1199454e83b48b9d1b0e879c/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:16:36.404830 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788eba9-616e-4c38-8b82-39da4a4303cb None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:36.406745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788eba9-616e-4c38-8b82-39da4a4303cb None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:36.407214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788eba9-616e-4c38-8b82-39da4a4303cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:36.494618 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9788eba9-616e-4c38-8b82-39da4a4303cb None None] MFA Rules not processed for user `0ab7eb2d1199454e83b48b9d1b0e879c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:36.501019 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9788eba9-616e-4c38-8b82-39da4a4303cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:36.618084 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1109/2209] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:36.632643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d516937-2a0b-4280-84cb-d5f4352545fc admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:36.634380 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2d516937-2a0b-4280-84cb-d5f4352545fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:36.709530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d516937-2a0b-4280-84cb-d5f4352545fc admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:36.716900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26de3e6-3bcb-4592-9e98-7174ba9de5a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:36.717557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26de3e6-3bcb-4592-9e98-7174ba9de5a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:36.717841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26de3e6-3bcb-4592-9e98-7174ba9de5a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:36.718120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26de3e6-3bcb-4592-9e98-7174ba9de5a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:36.736425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26de3e6-3bcb-4592-9e98-7174ba9de5a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:36.738635 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26de3e6-3bcb-4592-9e98-7174ba9de5a4 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:36.757462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1101/2210] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:41.168465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788eba9-616e-4c38-8b82-39da4a4303cb None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:41.278435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788eba9-616e-4c38-8b82-39da4a4303cb None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:41.285645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c194a2-5a08-4be9-bb7a-866db49cafb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:41.286443 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c194a2-5a08-4be9-bb7a-866db49cafb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:41.287008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c194a2-5a08-4be9-bb7a-866db49cafb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:41.287786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c194a2-5a08-4be9-bb7a-866db49cafb9 admin admin] PATH_INFO: `/v3/users/0ab7eb2d1199454e83b48b9d1b0e879c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:41.288793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c194a2-5a08-4be9-bb7a-866db49cafb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ab7eb2d1199454e83b48b9d1b0e879c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:41.291475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c194a2-5a08-4be9-bb7a-866db49cafb9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:41.365386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:41.377509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-48c194a2-5a08-4be9-bb7a-866db49cafb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ab7eb2d1199454e83b48b9d1b0e879c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:41.378332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-48c194a2-5a08-4be9-bb7a-866db49cafb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ab7eb2d1199454e83b48b9d1b0e879c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:41.390682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-48c194a2-5a08-4be9-bb7a-866db49cafb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ab7eb2d1199454e83b48b9d1b0e879c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:41.392513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-48c194a2-5a08-4be9-bb7a-866db49cafb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ab7eb2d1199454e83b48b9d1b0e879c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:41.408975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-48c194a2-5a08-4be9-bb7a-866db49cafb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ab7eb2d1199454e83b48b9d1b0e879c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:41.428011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1110/2211] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:16:41 2022] DELETE /identity/v3/users/0ab7eb2d1199454e83b48b9d1b0e879c => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:41.432584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284883f-c32b-4269-9c88-775b98976928 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:41.433174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284883f-c32b-4269-9c88-775b98976928 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:41.433174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284883f-c32b-4269-9c88-775b98976928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:41.455268 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1284883f-c32b-4269-9c88-775b98976928 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:41.461785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1284883f-c32b-4269-9c88-775b98976928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:41.542821 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1102/2212] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:41.553552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c194a2-5a08-4be9-bb7a-866db49cafb9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:41.555464 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-48c194a2-5a08-4be9-bb7a-866db49cafb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:41.644820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c194a2-5a08-4be9-bb7a-866db49cafb9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:41.649203 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed43f6-f032-4587-b29d-4c87a0593ae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:41.650226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed43f6-f032-4587-b29d-4c87a0593ae3 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:41.650226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed43f6-f032-4587-b29d-4c87a0593ae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:41.650436 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed43f6-f032-4587-b29d-4c87a0593ae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:41.661322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed43f6-f032-4587-b29d-4c87a0593ae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:41.662862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed43f6-f032-4587-b29d-4c87a0593ae3 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:41.675389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1111/2213] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:42.069815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284883f-c32b-4269-9c88-775b98976928 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:42.072144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1284883f-c32b-4269-9c88-775b98976928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:42.089624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284883f-c32b-4269-9c88-775b98976928 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:42.097156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc4573-7d44-4d44-b703-87ee2fdf0457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:42.098000 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc4573-7d44-4d44-b703-87ee2fdf0457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:42.099704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc4573-7d44-4d44-b703-87ee2fdf0457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:42.099999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc4573-7d44-4d44-b703-87ee2fdf0457 admin admin] PATH_INFO: `/v3/projects/9a171a0c8da74b03bc0b2450968005e7` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:42.101254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc4573-7d44-4d44-b703-87ee2fdf0457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a171a0c8da74b03bc0b2450968005e7)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:42.103337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc4573-7d44-4d44-b703-87ee2fdf0457 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:42.231022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:42.262128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0dfc4573-7d44-4d44-b703-87ee2fdf0457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a171a0c8da74b03bc0b2450968005e7', 'request_id': 'req-0dfc4573-7d44-4d44-b703-87ee2fdf0457'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:42.286163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0dfc4573-7d44-4d44-b703-87ee2fdf0457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a171a0c8da74b03bc0b2450968005e7', 'request_id': 'req-0dfc4573-7d44-4d44-b703-87ee2fdf0457'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:42.297370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0dfc4573-7d44-4d44-b703-87ee2fdf0457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a171a0c8da74b03bc0b2450968005e7', 'request_id': 'req-0dfc4573-7d44-4d44-b703-87ee2fdf0457'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:42.319941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0dfc4573-7d44-4d44-b703-87ee2fdf0457 admin admin] The token cache is being invalidate because project 9a171a0c8da74b03bc0b2450968005e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:42.320293 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0dfc4573-7d44-4d44-b703-87ee2fdf0457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:42.320793 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0dfc4573-7d44-4d44-b703-87ee2fdf0457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:42.322797 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1103/2214] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:16:42 2022] DELETE /identity/v3/projects/9a171a0c8da74b03bc0b2450968005e7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:42.330755 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38373641-cddf-4418-bba9-053f3edc72da None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:42.331555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38373641-cddf-4418-bba9-053f3edc72da None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:42.331555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38373641-cddf-4418-bba9-053f3edc72da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:42.352421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-38373641-cddf-4418-bba9-053f3edc72da None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:42.353639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-38373641-cddf-4418-bba9-053f3edc72da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:42.416864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1112/2215] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:42.426366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8a0a3-0922-4ee8-8484-15e0f1242336 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:42.426654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8a0a3-0922-4ee8-8484-15e0f1242336 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:42.426961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8a0a3-0922-4ee8-8484-15e0f1242336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:42.448649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-33c8a0a3-0922-4ee8-8484-15e0f1242336 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:42.449804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-33c8a0a3-0922-4ee8-8484-15e0f1242336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:42.459128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1104/2216] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:42.463753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38373641-cddf-4418-bba9-053f3edc72da None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:42.479033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38373641-cddf-4418-bba9-053f3edc72da None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:42.484028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7962e1-be0b-41d4-9317-db8c6b099781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:42.484653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7962e1-be0b-41d4-9317-db8c6b099781 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:42.484935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7962e1-be0b-41d4-9317-db8c6b099781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:42.485215 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7962e1-be0b-41d4-9317-db8c6b099781 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:42.485927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7962e1-be0b-41d4-9317-db8c6b099781 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:16:42.486237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7962e1-be0b-41d4-9317-db8c6b099781 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:42.487909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7962e1-be0b-41d4-9317-db8c6b099781 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:42.500929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1113/2217] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:16:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:16:42.505887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8a0a3-0922-4ee8-8484-15e0f1242336 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:42.516087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8a0a3-0922-4ee8-8484-15e0f1242336 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:42.524253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64471f2-add0-4b0f-bbf9-a7c56220e897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:42.524976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64471f2-add0-4b0f-bbf9-a7c56220e897 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:42.525259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64471f2-add0-4b0f-bbf9-a7c56220e897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:42.525532 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64471f2-add0-4b0f-bbf9-a7c56220e897 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:42.526218 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64471f2-add0-4b0f-bbf9-a7c56220e897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:42.528675 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64471f2-add0-4b0f-bbf9-a7c56220e897 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:42.559124 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1105/2218] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:16:42 2022] POST /identity/v3/projects => generated 380 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:42.563295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7962e1-be0b-41d4-9317-db8c6b099781 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:42.574142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7962e1-be0b-41d4-9317-db8c6b099781 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:42.622633 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:42.643038 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff5f1c-9f06-4c14-97c0-0501772ef6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:42.644081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff5f1c-9f06-4c14-97c0-0501772ef6eb admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:42.644769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff5f1c-9f06-4c14-97c0-0501772ef6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:42.645329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff5f1c-9f06-4c14-97c0-0501772ef6eb admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:42.647149 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff5f1c-9f06-4c14-97c0-0501772ef6eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:42.649637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff5f1c-9f06-4c14-97c0-0501772ef6eb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:42.678455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1114/2219] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:16:42 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:42.679555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64471f2-add0-4b0f-bbf9-a7c56220e897 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:42.695956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64471f2-add0-4b0f-bbf9-a7c56220e897 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:42.702109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127548cf-d14d-48d6-b8f6-c8b310029283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:42.702709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127548cf-d14d-48d6-b8f6-c8b310029283 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:42.703243 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127548cf-d14d-48d6-b8f6-c8b310029283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:42.703528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127548cf-d14d-48d6-b8f6-c8b310029283 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:42.704249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127548cf-d14d-48d6-b8f6-c8b310029283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:42.705884 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127548cf-d14d-48d6-b8f6-c8b310029283 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:42.718749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1106/2220] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:42 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:42.722892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff5f1c-9f06-4c14-97c0-0501772ef6eb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:42.741363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff5f1c-9f06-4c14-97c0-0501772ef6eb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:42.752206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1925108c-0fbe-451e-abe3-4481d2b17349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:42.752909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925108c-0fbe-451e-abe3-4481d2b17349 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:42.753310 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925108c-0fbe-451e-abe3-4481d2b17349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:42.753628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1925108c-0fbe-451e-abe3-4481d2b17349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:42.754511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1925108c-0fbe-451e-abe3-4481d2b17349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:42.756949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1925108c-0fbe-451e-abe3-4481d2b17349 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:42.775195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1115/2221] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:42 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:42.781903 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127548cf-d14d-48d6-b8f6-c8b310029283 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:42.800821 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127548cf-d14d-48d6-b8f6-c8b310029283 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:42.821597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711d5bc-c091-440b-a612-b51488a12ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:42.822644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711d5bc-c091-440b-a612-b51488a12ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:42.825287 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711d5bc-c091-440b-a612-b51488a12ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:42.825484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711d5bc-c091-440b-a612-b51488a12ac7 admin admin] PATH_INFO: `/v3/projects/f44ad042ba9b4bdc91783fce54774c29/users/c6795a827b8d411aaa165db7b38435ce/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:42.858335 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711d5bc-c091-440b-a612-b51488a12ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f44ad042ba9b4bdc91783fce54774c29, user_id=c6795a827b8d411aaa165db7b38435ce, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:42.861942 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711d5bc-c091-440b-a612-b51488a12ac7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:42.881048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1107/2222] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:42 2022] PUT /identity/v3/projects/f44ad042ba9b4bdc91783fce54774c29/users/c6795a827b8d411aaa165db7b38435ce/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:42.885826 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b2f1d-d372-47ab-a87c-e09a8f99ff8f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:42.886116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b2f1d-d372-47ab-a87c-e09a8f99ff8f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:42.887134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b2f1d-d372-47ab-a87c-e09a8f99ff8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:42.908715 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-358b2f1d-d372-47ab-a87c-e09a8f99ff8f None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:42.909951 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-358b2f1d-d372-47ab-a87c-e09a8f99ff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:43.035388 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1116/2223] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:43.048342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711d5bc-c091-440b-a612-b51488a12ac7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:43.050238 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c711d5bc-c091-440b-a612-b51488a12ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:43.177883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711d5bc-c091-440b-a612-b51488a12ac7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:43.185525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f06c3-2b33-4271-bb97-a3a94b991ed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:43.186356 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f06c3-2b33-4271-bb97-a3a94b991ed0 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:43.186903 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f06c3-2b33-4271-bb97-a3a94b991ed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:43.187677 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f06c3-2b33-4271-bb97-a3a94b991ed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:43.216043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f06c3-2b33-4271-bb97-a3a94b991ed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:43.217546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f06c3-2b33-4271-bb97-a3a94b991ed0 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:43.239687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1108/2224] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:43.590971 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b2f1d-d372-47ab-a87c-e09a8f99ff8f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:43.592674 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-358b2f1d-d372-47ab-a87c-e09a8f99ff8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:43.605583 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b2f1d-d372-47ab-a87c-e09a8f99ff8f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:43.610156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd198451-bf0f-4679-ba99-30671e19b840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:43.610882 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd198451-bf0f-4679-ba99-30671e19b840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:43.611166 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd198451-bf0f-4679-ba99-30671e19b840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:43.611437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd198451-bf0f-4679-ba99-30671e19b840 admin admin] PATH_INFO: `/v3/users/e8fc60410fab43a39bdb80478348ddae` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:43.612665 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd198451-bf0f-4679-ba99-30671e19b840 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8fc60410fab43a39bdb80478348ddae)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:43.615229 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd198451-bf0f-4679-ba99-30671e19b840 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:43.671105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bd198451-bf0f-4679-ba99-30671e19b840 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8fc60410fab43a39bdb80478348ddae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:43.671809 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bd198451-bf0f-4679-ba99-30671e19b840 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8fc60410fab43a39bdb80478348ddae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:43.684573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bd198451-bf0f-4679-ba99-30671e19b840 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8fc60410fab43a39bdb80478348ddae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:43.685318 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bd198451-bf0f-4679-ba99-30671e19b840 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8fc60410fab43a39bdb80478348ddae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:43.702508 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bd198451-bf0f-4679-ba99-30671e19b840 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8fc60410fab43a39bdb80478348ddae'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:43.720322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1117/2225] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:16:43 2022] DELETE /identity/v3/users/e8fc60410fab43a39bdb80478348ddae => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:43.943713 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f06c3-2b33-4271-bb97-a3a94b991ed0 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:43.971163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f06c3-2b33-4271-bb97-a3a94b991ed0 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:44.038231 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:44.049587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:44.050274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.050565 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.050953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7 admin admin] PATH_INFO: `/v3/projects/85a7e64a4f0a43599f9eeeb99e695f24` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.052887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a7e64a4f0a43599f9eeeb99e695f24)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:44.054904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:44.128320 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a7e64a4f0a43599f9eeeb99e695f24', 'request_id': 'req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:44.135823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a7e64a4f0a43599f9eeeb99e695f24', 'request_id': 'req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:44.144444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a7e64a4f0a43599f9eeeb99e695f24', 'request_id': 'req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:44.155869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7 admin admin] The token cache is being invalidate because project 85a7e64a4f0a43599f9eeeb99e695f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:16:44.156297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:44.157080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9fe6b2b9-2a8d-4d24-aff5-98d46c0e91f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:16:44.158617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1109/2226] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:16:43 2022] DELETE /identity/v3/projects/85a7e64a4f0a43599f9eeeb99e695f24 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:44.169784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029f788-1fab-42f1-a870-a6fad6536c81 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.170069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029f788-1fab-42f1-a870-a6fad6536c81 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.170416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029f788-1fab-42f1-a870-a6fad6536c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.193179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d029f788-1fab-42f1-a870-a6fad6536c81 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:44.194439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d029f788-1fab-42f1-a870-a6fad6536c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:44.268818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:44.386351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1118/2227] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:44.409278 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852f2de-541b-44dc-a6da-0983f5675796 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.410900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852f2de-541b-44dc-a6da-0983f5675796 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.411603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852f2de-541b-44dc-a6da-0983f5675796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.464380 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d852f2de-541b-44dc-a6da-0983f5675796 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:44.465824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d852f2de-541b-44dc-a6da-0983f5675796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:44.482513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029f788-1fab-42f1-a870-a6fad6536c81 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:44.493412 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1110/2228] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:44.494637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029f788-1fab-42f1-a870-a6fad6536c81 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:44.501447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8030137-c7bb-42d4-b2b2-88f4597119ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:44.502265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8030137-c7bb-42d4-b2b2-88f4597119ed admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.503013 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8030137-c7bb-42d4-b2b2-88f4597119ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.503013 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8030137-c7bb-42d4-b2b2-88f4597119ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.503802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8030137-c7bb-42d4-b2b2-88f4597119ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:16:44.504103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8030137-c7bb-42d4-b2b2-88f4597119ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:44.505883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8030137-c7bb-42d4-b2b2-88f4597119ed admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:44.526627 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1119/2229] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:16:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:16:44.531574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852f2de-541b-44dc-a6da-0983f5675796 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:44.550900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852f2de-541b-44dc-a6da-0983f5675796 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:44.556922 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4fd61-4777-4b94-af90-2a99040bccf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:44.557624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4fd61-4777-4b94-af90-2a99040bccf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.558010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4fd61-4777-4b94-af90-2a99040bccf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.558290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4fd61-4777-4b94-af90-2a99040bccf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.561404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4fd61-4777-4b94-af90-2a99040bccf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:44.565765 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4fd61-4777-4b94-af90-2a99040bccf9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:44.633877 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1111/2230] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:16:44 2022] POST /identity/v3/projects => generated 364 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:44.638470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8030137-c7bb-42d4-b2b2-88f4597119ed admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:44.655032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8030137-c7bb-42d4-b2b2-88f4597119ed admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:44.755978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58069005-95a7-4d20-862c-d3fb19f4ddbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:44.757520 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58069005-95a7-4d20-862c-d3fb19f4ddbf admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.759221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58069005-95a7-4d20-862c-d3fb19f4ddbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.759438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58069005-95a7-4d20-862c-d3fb19f4ddbf admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.760152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58069005-95a7-4d20-862c-d3fb19f4ddbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:44.763815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58069005-95a7-4d20-862c-d3fb19f4ddbf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:44.811159 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1120/2231] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:16:44 2022] POST /identity/v3/users => generated 283 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:16:44.815433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4fd61-4777-4b94-af90-2a99040bccf9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:44.830789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4fd61-4777-4b94-af90-2a99040bccf9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:44.836759 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf715261-a32b-43b3-9d80-7412fde5c3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:44.837515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf715261-a32b-43b3-9d80-7412fde5c3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.838049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf715261-a32b-43b3-9d80-7412fde5c3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.838339 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf715261-a32b-43b3-9d80-7412fde5c3b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.839207 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf715261-a32b-43b3-9d80-7412fde5c3b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:44.840915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf715261-a32b-43b3-9d80-7412fde5c3b9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:44.853021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1112/2232] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:44 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:44.856764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58069005-95a7-4d20-862c-d3fb19f4ddbf admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:44.871603 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58069005-95a7-4d20-862c-d3fb19f4ddbf admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:44.891062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52237908-a081-4757-bef3-c1fe4dc73dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:44.892325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52237908-a081-4757-bef3-c1fe4dc73dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.892735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52237908-a081-4757-bef3-c1fe4dc73dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.893081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52237908-a081-4757-bef3-c1fe4dc73dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.893880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52237908-a081-4757-bef3-c1fe4dc73dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:44.895663 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52237908-a081-4757-bef3-c1fe4dc73dfb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:44.915736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1121/2233] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:44 2022] GET /identity/v3/roles => generated 1427 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:16:44.916433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf715261-a32b-43b3-9d80-7412fde5c3b9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:44.929201 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf715261-a32b-43b3-9d80-7412fde5c3b9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:44.935313 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2e173-3386-4741-ae9e-f18b66d56646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:44.936241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2e173-3386-4741-ae9e-f18b66d56646 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.936490 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2e173-3386-4741-ae9e-f18b66d56646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.936849 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2e173-3386-4741-ae9e-f18b66d56646 admin admin] PATH_INFO: `/v3/projects/646b866e38b8495a98d3d6716d667a53/users/aa9200efec3d4284a46b754cb3a49f2d/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:44.962926 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2e173-3386-4741-ae9e-f18b66d56646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646b866e38b8495a98d3d6716d667a53, user_id=aa9200efec3d4284a46b754cb3a49f2d, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:44.966333 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2e173-3386-4741-ae9e-f18b66d56646 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:44.986083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1113/2234] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:44 2022] PUT /identity/v3/projects/646b866e38b8495a98d3d6716d667a53/users/aa9200efec3d4284a46b754cb3a49f2d/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:44.998924 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba915745-8b67-4fe3-803b-fe345cffa017 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:44.999216 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba915745-8b67-4fe3-803b-fe345cffa017 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:44.999498 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba915745-8b67-4fe3-803b-fe345cffa017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:45.129846 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ba915745-8b67-4fe3-803b-fe345cffa017 None None] MFA Rules not processed for user `aa9200efec3d4284a46b754cb3a49f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:45.133261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ba915745-8b67-4fe3-803b-fe345cffa017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:45.303875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1122/2235] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:45.304695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2e173-3386-4741-ae9e-f18b66d56646 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:45.402558 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2e173-3386-4741-ae9e-f18b66d56646 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:45.407688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f5959-6edb-4b0d-8d3c-d1a4212ed3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:45.408327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f5959-6edb-4b0d-8d3c-d1a4212ed3c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:45.408893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f5959-6edb-4b0d-8d3c-d1a4212ed3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:45.409323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f5959-6edb-4b0d-8d3c-d1a4212ed3c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:45.410177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f5959-6edb-4b0d-8d3c-d1a4212ed3c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:45.412381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f5959-6edb-4b0d-8d3c-d1a4212ed3c5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:45.450783 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1114/2236] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:16:45 2022] POST /identity/v3/projects => generated 362 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:45.454697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba915745-8b67-4fe3-803b-fe345cffa017 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:45.466284 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba915745-8b67-4fe3-803b-fe345cffa017 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:45.580302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:45.628822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f723831-2009-4a95-bfec-3fcbe2d89a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:45.629468 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f723831-2009-4a95-bfec-3fcbe2d89a70 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:45.629749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f723831-2009-4a95-bfec-3fcbe2d89a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:45.630028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f723831-2009-4a95-bfec-3fcbe2d89a70 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:45.630911 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f723831-2009-4a95-bfec-3fcbe2d89a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:45.633685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f723831-2009-4a95-bfec-3fcbe2d89a70 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:45.663569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1123/2237] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:16:45 2022] POST /identity/v3/users => generated 288 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:45.667508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f5959-6edb-4b0d-8d3c-d1a4212ed3c5 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:45.680607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:45.690269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f5959-6edb-4b0d-8d3c-d1a4212ed3c5 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:45.702477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00204e5-90fc-45f1-9d4c-a272340ed5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:45.703279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00204e5-90fc-45f1-9d4c-a272340ed5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:45.703632 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00204e5-90fc-45f1-9d4c-a272340ed5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:45.704587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00204e5-90fc-45f1-9d4c-a272340ed5a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:45.705140 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00204e5-90fc-45f1-9d4c-a272340ed5a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:45.718880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00204e5-90fc-45f1-9d4c-a272340ed5a0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:45.737392 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1115/2238] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:45 2022] GET /identity/v3/roles => generated 1427 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:45.750789 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f723831-2009-4a95-bfec-3fcbe2d89a70 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:45.767539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f723831-2009-4a95-bfec-3fcbe2d89a70 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:45.775038 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9187cd-ad2e-4574-ab60-522b0ccf3ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:45.775812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9187cd-ad2e-4574-ab60-522b0ccf3ef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:45.776068 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9187cd-ad2e-4574-ab60-522b0ccf3ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:45.776355 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9187cd-ad2e-4574-ab60-522b0ccf3ef9 admin admin] PATH_INFO: `/v3/projects/734f7075d4ca4b738eff00739c1911db/users/7bee1fd96d06468aa50ce8b7dd4dbacd/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:45.801633 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9187cd-ad2e-4574-ab60-522b0ccf3ef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=734f7075d4ca4b738eff00739c1911db, user_id=7bee1fd96d06468aa50ce8b7dd4dbacd, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:45.806170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9187cd-ad2e-4574-ab60-522b0ccf3ef9 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:45.832650 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1124/2239] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:45 2022] PUT /identity/v3/projects/734f7075d4ca4b738eff00739c1911db/users/7bee1fd96d06468aa50ce8b7dd4dbacd/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:45.837311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00204e5-90fc-45f1-9d4c-a272340ed5a0 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:45.849771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00204e5-90fc-45f1-9d4c-a272340ed5a0 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:45.985519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db0fbd-53be-4ba9-addb-aad4f79140af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:45.987103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db0fbd-53be-4ba9-addb-aad4f79140af admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:45.987653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db0fbd-53be-4ba9-addb-aad4f79140af admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:45.988143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db0fbd-53be-4ba9-addb-aad4f79140af admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:45.989086 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db0fbd-53be-4ba9-addb-aad4f79140af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:45.994148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db0fbd-53be-4ba9-addb-aad4f79140af admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:46.008784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1116/2240] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:45 2022] GET /identity/v3/roles => generated 1427 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:46.012865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9187cd-ad2e-4574-ab60-522b0ccf3ef9 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:46.031368 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9187cd-ad2e-4574-ab60-522b0ccf3ef9 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:46.037244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfafa3-829a-4357-8996-899436ff4fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:46.038011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfafa3-829a-4357-8996-899436ff4fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:46.038389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfafa3-829a-4357-8996-899436ff4fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:46.039375 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfafa3-829a-4357-8996-899436ff4fd7 admin admin] PATH_INFO: `/v3/projects/734f7075d4ca4b738eff00739c1911db/users/7bee1fd96d06468aa50ce8b7dd4dbacd/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:46.042852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfafa3-829a-4357-8996-899436ff4fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=734f7075d4ca4b738eff00739c1911db, user_id=7bee1fd96d06468aa50ce8b7dd4dbacd, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:46.045311 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dfafa3-829a-4357-8996-899436ff4fd7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:46.068446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209c53c-8201-4b65-970b-1f0fb8b157f9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:46.068739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209c53c-8201-4b65-970b-1f0fb8b157f9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:46.069025 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209c53c-8201-4b65-970b-1f0fb8b157f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:46.070770 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1125/2241] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:46 2022] PUT /identity/v3/projects/734f7075d4ca4b738eff00739c1911db/users/7bee1fd96d06468aa50ce8b7dd4dbacd/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:46.108950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e209c53c-8201-4b65-970b-1f0fb8b157f9 None None] MFA Rules not processed for user `7bee1fd96d06468aa50ce8b7dd4dbacd`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:46.110207 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e209c53c-8201-4b65-970b-1f0fb8b157f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:46.257715 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1117/2242] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:46 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:46.271013 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfafa3-829a-4357-8996-899436ff4fd7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:46.273049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-95dfafa3-829a-4357-8996-899436ff4fd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:46.395587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfafa3-829a-4357-8996-899436ff4fd7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:46.399635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cada33-cc4b-42d0-a8cc-085379e5d563 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:46.401254 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cada33-cc4b-42d0-a8cc-085379e5d563 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:46.401542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cada33-cc4b-42d0-a8cc-085379e5d563 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:46.424308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cada33-cc4b-42d0-a8cc-085379e5d563 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:46.441168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cada33-cc4b-42d0-a8cc-085379e5d563 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:46.443286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cada33-cc4b-42d0-a8cc-085379e5d563 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:46.456859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1126/2243] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:46 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:46.548930 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209c53c-8201-4b65-970b-1f0fb8b157f9 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:46.560331 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209c53c-8201-4b65-970b-1f0fb8b157f9 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:46.565194 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f6f55-7445-428a-b4ea-6463a64c1a01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:46.565194 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f6f55-7445-428a-b4ea-6463a64c1a01 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:46.570192 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f6f55-7445-428a-b4ea-6463a64c1a01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:46.570192 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f6f55-7445-428a-b4ea-6463a64c1a01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:46.687351 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f6f55-7445-428a-b4ea-6463a64c1a01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:46.690649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f6f55-7445-428a-b4ea-6463a64c1a01 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:46.707778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1118/2244] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:16:46.857300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d470e20-a0c0-4229-b2e6-17324be1f2d4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:46.858148 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d470e20-a0c0-4229-b2e6-17324be1f2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:46.858569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d470e20-a0c0-4229-b2e6-17324be1f2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:46.881361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6d470e20-a0c0-4229-b2e6-17324be1f2d4 None None] MFA Rules not processed for user `55bb7887d98f458688cffbccf4a64b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:46.882956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6d470e20-a0c0-4229-b2e6-17324be1f2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:46.968849 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1127/2245] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:49.650214 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae60019-90a2-425a-94c1-d3709a163f44 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:49.650848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae60019-90a2-425a-94c1-d3709a163f44 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:49.651337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae60019-90a2-425a-94c1-d3709a163f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:49.734635 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:49.737080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3ae60019-90a2-425a-94c1-d3709a163f44 None None] MFA Rules not processed for user `c6795a827b8d411aaa165db7b38435ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:49.739964 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3ae60019-90a2-425a-94c1-d3709a163f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:49.915103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1119/2246] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:49.917406 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d470e20-a0c0-4229-b2e6-17324be1f2d4 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:49.919157 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6d470e20-a0c0-4229-b2e6-17324be1f2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:49.974981 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:50.018164 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d470e20-a0c0-4229-b2e6-17324be1f2d4 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:50.024439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace08af5-f16d-4948-8b63-ed1fb01b4e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:50.025366 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace08af5-f16d-4948-8b63-ed1fb01b4e85 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:50.025914 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace08af5-f16d-4948-8b63-ed1fb01b4e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:50.026342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace08af5-f16d-4948-8b63-ed1fb01b4e85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:50.028108 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace08af5-f16d-4948-8b63-ed1fb01b4e85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:50.032561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace08af5-f16d-4948-8b63-ed1fb01b4e85 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:50.090802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1128/2247] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:16:49 2022] POST /identity/v3/projects => generated 376 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:50.094310 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae60019-90a2-425a-94c1-d3709a163f44 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:50.108261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae60019-90a2-425a-94c1-d3709a163f44 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:50.237271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3b192-8015-4dd6-b9d6-426c71955e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:50.238321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3b192-8015-4dd6-b9d6-426c71955e70 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:50.239046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3b192-8015-4dd6-b9d6-426c71955e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:50.240281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3b192-8015-4dd6-b9d6-426c71955e70 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:50.241190 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3b192-8015-4dd6-b9d6-426c71955e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:50.243747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3b192-8015-4dd6-b9d6-426c71955e70 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:50.279848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1120/2248] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:16:50 2022] POST /identity/v3/users => generated 295 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:16:50.283429 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace08af5-f16d-4948-8b63-ed1fb01b4e85 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:50.305873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace08af5-f16d-4948-8b63-ed1fb01b4e85 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:50.314451 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe7394-a383-41a6-9f41-1709ddf52547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:50.316397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe7394-a383-41a6-9f41-1709ddf52547 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:50.316908 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe7394-a383-41a6-9f41-1709ddf52547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:50.317398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe7394-a383-41a6-9f41-1709ddf52547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:50.318421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe7394-a383-41a6-9f41-1709ddf52547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:50.320708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe7394-a383-41a6-9f41-1709ddf52547 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:50.345517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1129/2249] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:50 2022] GET /identity/v3/roles => generated 1427 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:50.350471 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3b192-8015-4dd6-b9d6-426c71955e70 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:50.376374 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3b192-8015-4dd6-b9d6-426c71955e70 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:50.390983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ec6d0-54d8-4e57-9f49-6f538b4e5ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:50.391687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ec6d0-54d8-4e57-9f49-6f538b4e5ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:50.391972 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ec6d0-54d8-4e57-9f49-6f538b4e5ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:50.392274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ec6d0-54d8-4e57-9f49-6f538b4e5ddb admin admin] PATH_INFO: `/v3/projects/2371e97d1d20446db4e42d117ff84937/users/2245e963d3c44a5c8aea0d34cd0c4e9e/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:50.428805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ec6d0-54d8-4e57-9f49-6f538b4e5ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2371e97d1d20446db4e42d117ff84937, user_id=2245e963d3c44a5c8aea0d34cd0c4e9e, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:50.433359 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ec6d0-54d8-4e57-9f49-6f538b4e5ddb admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:50.467233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1121/2250] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:50 2022] PUT /identity/v3/projects/2371e97d1d20446db4e42d117ff84937/users/2245e963d3c44a5c8aea0d34cd0c4e9e/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:50.472536 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe7394-a383-41a6-9f41-1709ddf52547 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:50.484776 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe7394-a383-41a6-9f41-1709ddf52547 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:50.555059 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c6497-ed3d-4cc1-879f-5cb5fc3c4c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:50.555852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c6497-ed3d-4cc1-879f-5cb5fc3c4c4a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:50.556337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c6497-ed3d-4cc1-879f-5cb5fc3c4c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:50.556677 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c6497-ed3d-4cc1-879f-5cb5fc3c4c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:50.557489 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c6497-ed3d-4cc1-879f-5cb5fc3c4c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:50.559328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c6497-ed3d-4cc1-879f-5cb5fc3c4c4a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:50.571424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1130/2251] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:16:50 2022] GET /identity/v3/roles => generated 1427 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:16:50.575375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ec6d0-54d8-4e57-9f49-6f538b4e5ddb admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:50.589378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ec6d0-54d8-4e57-9f49-6f538b4e5ddb admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:50.594931 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe43e3-9fac-4f38-8235-7c41cbfeae21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:50.596237 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe43e3-9fac-4f38-8235-7c41cbfeae21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:50.596237 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe43e3-9fac-4f38-8235-7c41cbfeae21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:50.596591 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe43e3-9fac-4f38-8235-7c41cbfeae21 admin admin] PATH_INFO: `/v3/projects/2371e97d1d20446db4e42d117ff84937/users/2245e963d3c44a5c8aea0d34cd0c4e9e/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:50.599386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe43e3-9fac-4f38-8235-7c41cbfeae21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2371e97d1d20446db4e42d117ff84937, user_id=2245e963d3c44a5c8aea0d34cd0c4e9e, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:50.602183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe43e3-9fac-4f38-8235-7c41cbfeae21 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:50.619814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1122/2252] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:16:50 2022] PUT /identity/v3/projects/2371e97d1d20446db4e42d117ff84937/users/2245e963d3c44a5c8aea0d34cd0c4e9e/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:16:56.507411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc2ab1-6649-43b3-91b0-a1f39bbda809 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:56.508030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc2ab1-6649-43b3-91b0-a1f39bbda809 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:56.508030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc2ab1-6649-43b3-91b0-a1f39bbda809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:56.541520 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-08bc2ab1-6649-43b3-91b0-a1f39bbda809 None None] MFA Rules not processed for user `2245e963d3c44a5c8aea0d34cd0c4e9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:16:56.543406 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-08bc2ab1-6649-43b3-91b0-a1f39bbda809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:56.639863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1131/2253] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:16:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:16:56.661729 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe43e3-9fac-4f38-8235-7c41cbfeae21 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:16:56.663663 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7fbe43e3-9fac-4f38-8235-7c41cbfeae21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:16:56.726387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:16:56.749156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe43e3-9fac-4f38-8235-7c41cbfeae21 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:16:56.769375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368f08d-a47c-44af-8cd9-46cf2e516865 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:16:56.770037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368f08d-a47c-44af-8cd9-46cf2e516865 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:16:56.770379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368f08d-a47c-44af-8cd9-46cf2e516865 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:16:56.771356 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368f08d-a47c-44af-8cd9-46cf2e516865 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:16:56.845728 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368f08d-a47c-44af-8cd9-46cf2e516865 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:16:56.847471 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368f08d-a47c-44af-8cd9-46cf2e516865 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:16:56.859151 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1123/2254] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:16:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:17:03.497587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc2ab1-6649-43b3-91b0-a1f39bbda809 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:03.529499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:03.568454 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc2ab1-6649-43b3-91b0-a1f39bbda809 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:03.575980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a2585-1f43-48ba-9172-35c799bb7745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:03.576664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a2585-1f43-48ba-9172-35c799bb7745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:03.576961 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a2585-1f43-48ba-9172-35c799bb7745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:03.577253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a2585-1f43-48ba-9172-35c799bb7745 admin admin] PATH_INFO: `/v3/users/aa9200efec3d4284a46b754cb3a49f2d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:03.579240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a2585-1f43-48ba-9172-35c799bb7745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa9200efec3d4284a46b754cb3a49f2d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:03.583294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a2585-1f43-48ba-9172-35c799bb7745 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:03.645031 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c44a2585-1f43-48ba-9172-35c799bb7745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa9200efec3d4284a46b754cb3a49f2d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:03.645894 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c44a2585-1f43-48ba-9172-35c799bb7745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa9200efec3d4284a46b754cb3a49f2d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:03.655658 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c44a2585-1f43-48ba-9172-35c799bb7745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa9200efec3d4284a46b754cb3a49f2d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:03.656446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c44a2585-1f43-48ba-9172-35c799bb7745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa9200efec3d4284a46b754cb3a49f2d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:03.669742 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c44a2585-1f43-48ba-9172-35c799bb7745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa9200efec3d4284a46b754cb3a49f2d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:03.684885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1132/2255] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:17:03 2022] DELETE /identity/v3/users/aa9200efec3d4284a46b754cb3a49f2d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:03.689798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8415b4-9929-4a7a-b671-980637c9934e None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:03.690359 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8415b4-9929-4a7a-b671-980637c9934e None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:03.690470 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8415b4-9929-4a7a-b671-980637c9934e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:03.715954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-6d8415b4-9929-4a7a-b671-980637c9934e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:03.715954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6d8415b4-9929-4a7a-b671-980637c9934e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:03.772786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1124/2256] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:03.782524 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a2585-1f43-48ba-9172-35c799bb7745 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:03.784046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c44a2585-1f43-48ba-9172-35c799bb7745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:03.845667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a2585-1f43-48ba-9172-35c799bb7745 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:03.850162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a72af4-abe6-4f51-9afc-eeba3d363d54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:03.850833 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a72af4-abe6-4f51-9afc-eeba3d363d54 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:03.851119 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a72af4-abe6-4f51-9afc-eeba3d363d54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:03.851477 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a72af4-abe6-4f51-9afc-eeba3d363d54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:03.863533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a72af4-abe6-4f51-9afc-eeba3d363d54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:03.865015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a72af4-abe6-4f51-9afc-eeba3d363d54 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:03.876069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1133/2257] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:04.182626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8415b4-9929-4a7a-b671-980637c9934e None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:04.184121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6d8415b4-9929-4a7a-b671-980637c9934e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:04.198557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8415b4-9929-4a7a-b671-980637c9934e None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:04.203640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:04.204297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:04.204581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:04.204862 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] PATH_INFO: `/v3/projects/646b866e38b8495a98d3d6716d667a53` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:04.206394 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=646b866e38b8495a98d3d6716d667a53)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:04.209398 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:04.260611 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646b866e38b8495a98d3d6716d667a53', 'request_id': 'req-56d36b85-7b6b-48e7-8eac-b782d4759e48'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:04.267381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646b866e38b8495a98d3d6716d667a53', 'request_id': 'req-56d36b85-7b6b-48e7-8eac-b782d4759e48'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:04.276620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646b866e38b8495a98d3d6716d667a53', 'request_id': 'req-56d36b85-7b6b-48e7-8eac-b782d4759e48'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:04.287893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] The token cache is being invalidate because project 646b866e38b8495a98d3d6716d667a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:17:04.288192 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:04.288863 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:04.290483 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1125/2258] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:17:04 2022] DELETE /identity/v3/projects/646b866e38b8495a98d3d6716d667a53 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:04.296004 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a72af4-abe6-4f51-9afc-eeba3d363d54 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:04.305901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a72af4-abe6-4f51-9afc-eeba3d363d54 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:04.363655 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:04.375788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18524b26-4e53-4478-8224-287861749f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:04.376449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18524b26-4e53-4478-8224-287861749f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:04.376735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18524b26-4e53-4478-8224-287861749f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:04.377012 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18524b26-4e53-4478-8224-287861749f3e admin admin] PATH_INFO: `/v3/users/7bee1fd96d06468aa50ce8b7dd4dbacd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:04.378213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18524b26-4e53-4478-8224-287861749f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bee1fd96d06468aa50ce8b7dd4dbacd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:04.383267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18524b26-4e53-4478-8224-287861749f3e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:04.455494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-18524b26-4e53-4478-8224-287861749f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bee1fd96d06468aa50ce8b7dd4dbacd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:04.456491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-18524b26-4e53-4478-8224-287861749f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bee1fd96d06468aa50ce8b7dd4dbacd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:04.466694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-18524b26-4e53-4478-8224-287861749f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bee1fd96d06468aa50ce8b7dd4dbacd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:04.467343 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-18524b26-4e53-4478-8224-287861749f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bee1fd96d06468aa50ce8b7dd4dbacd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:04.498263 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-18524b26-4e53-4478-8224-287861749f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bee1fd96d06468aa50ce8b7dd4dbacd'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:04.521300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1134/2259] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:17:04 2022] DELETE /identity/v3/users/7bee1fd96d06468aa50ce8b7dd4dbacd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:04.852144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:04.854581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:04.872687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d36b85-7b6b-48e7-8eac-b782d4759e48 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:04.888932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:04.944498 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ac4d6-00c4-44a7-8830-1a1802abc297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:04.945255 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ac4d6-00c4-44a7-8830-1a1802abc297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:04.945691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ac4d6-00c4-44a7-8830-1a1802abc297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:04.945967 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ac4d6-00c4-44a7-8830-1a1802abc297 admin admin] PATH_INFO: `/v3/projects/734f7075d4ca4b738eff00739c1911db` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:04.947241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ac4d6-00c4-44a7-8830-1a1802abc297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=734f7075d4ca4b738eff00739c1911db)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:04.951557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ac4d6-00c4-44a7-8830-1a1802abc297 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:05.001641 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ee0ac4d6-00c4-44a7-8830-1a1802abc297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '734f7075d4ca4b738eff00739c1911db', 'request_id': 'req-ee0ac4d6-00c4-44a7-8830-1a1802abc297'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:05.013310 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ee0ac4d6-00c4-44a7-8830-1a1802abc297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '734f7075d4ca4b738eff00739c1911db', 'request_id': 'req-ee0ac4d6-00c4-44a7-8830-1a1802abc297'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:05.023814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ee0ac4d6-00c4-44a7-8830-1a1802abc297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '734f7075d4ca4b738eff00739c1911db', 'request_id': 'req-ee0ac4d6-00c4-44a7-8830-1a1802abc297'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:05.036688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ee0ac4d6-00c4-44a7-8830-1a1802abc297 admin admin] The token cache is being invalidate because project 734f7075d4ca4b738eff00739c1911db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:17:05.037049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ee0ac4d6-00c4-44a7-8830-1a1802abc297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:05.037701 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ee0ac4d6-00c4-44a7-8830-1a1802abc297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:05.039421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1126/2260] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:17:04 2022] DELETE /identity/v3/projects/734f7075d4ca4b738eff00739c1911db => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:05.045372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bfc36-fc08-4555-b702-3cc80076607e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.045613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bfc36-fc08-4555-b702-3cc80076607e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.045909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bfc36-fc08-4555-b702-3cc80076607e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.067675 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-149bfc36-fc08-4555-b702-3cc80076607e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:05.068911 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-149bfc36-fc08-4555-b702-3cc80076607e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:05.130155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1135/2261] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:05.138281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ac44-b58d-49e2-9298-a9e981b34aa8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.138574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ac44-b58d-49e2-9298-a9e981b34aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.138892 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48ac44-b58d-49e2-9298-a9e981b34aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.159795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-fd48ac44-b58d-49e2-9298-a9e981b34aa8 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:05.161041 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fd48ac44-b58d-49e2-9298-a9e981b34aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:05.169171 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1127/2262] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:05.173596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bfc36-fc08-4555-b702-3cc80076607e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:05.183428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bfc36-fc08-4555-b702-3cc80076607e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:05.188047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b702cc-7f16-4f40-9562-0c174bb27e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:05.188659 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b702cc-7f16-4f40-9562-0c174bb27e33 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.188943 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b702cc-7f16-4f40-9562-0c174bb27e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.189220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b702cc-7f16-4f40-9562-0c174bb27e33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.189944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b702cc-7f16-4f40-9562-0c174bb27e33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:17:05.190248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b702cc-7f16-4f40-9562-0c174bb27e33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:05.192412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b702cc-7f16-4f40-9562-0c174bb27e33 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:05.204019 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1136/2263] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:17:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:17:05.210329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48ac44-b58d-49e2-9298-a9e981b34aa8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:05.219982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48ac44-b58d-49e2-9298-a9e981b34aa8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:05.224747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fc259-f2a5-4846-9120-8a825e90f9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:05.225363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fc259-f2a5-4846-9120-8a825e90f9a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.225650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fc259-f2a5-4846-9120-8a825e90f9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.225930 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fc259-f2a5-4846-9120-8a825e90f9a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.226614 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fc259-f2a5-4846-9120-8a825e90f9a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:05.228573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fc259-f2a5-4846-9120-8a825e90f9a5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:05.260110 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1128/2264] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:17:05 2022] POST /identity/v3/projects => generated 392 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:17:05.264479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b702cc-7f16-4f40-9562-0c174bb27e33 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:05.275485 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b702cc-7f16-4f40-9562-0c174bb27e33 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:05.345489 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:05.352586 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e55f4-f289-4de4-aaf0-bbc77578bf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:05.353551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e55f4-f289-4de4-aaf0-bbc77578bf84 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.353841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e55f4-f289-4de4-aaf0-bbc77578bf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.354459 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e55f4-f289-4de4-aaf0-bbc77578bf84 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.355244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e55f4-f289-4de4-aaf0-bbc77578bf84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:05.357164 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e55f4-f289-4de4-aaf0-bbc77578bf84 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:05.381859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1137/2265] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:17:05 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:17:05.386228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fc259-f2a5-4846-9120-8a825e90f9a5 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:05.396760 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fc259-f2a5-4846-9120-8a825e90f9a5 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:05.401546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a9a44-35a3-45be-93ab-1cd30255ddc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:05.402149 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a9a44-35a3-45be-93ab-1cd30255ddc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.402429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a9a44-35a3-45be-93ab-1cd30255ddc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.402738 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a9a44-35a3-45be-93ab-1cd30255ddc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.403442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a9a44-35a3-45be-93ab-1cd30255ddc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:05.405022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a9a44-35a3-45be-93ab-1cd30255ddc1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:05.415693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1129/2266] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:05 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:17:05.419803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e55f4-f289-4de4-aaf0-bbc77578bf84 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:05.431590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e55f4-f289-4de4-aaf0-bbc77578bf84 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:05.436660 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e442425-9312-4b5b-ae07-1b1642aaf74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:05.437270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442425-9312-4b5b-ae07-1b1642aaf74e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.437552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442425-9312-4b5b-ae07-1b1642aaf74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.437829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e442425-9312-4b5b-ae07-1b1642aaf74e admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.438531 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e442425-9312-4b5b-ae07-1b1642aaf74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:05.440203 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e442425-9312-4b5b-ae07-1b1642aaf74e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:05.451583 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1138/2267] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:05 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:17:05.456436 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a9a44-35a3-45be-93ab-1cd30255ddc1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:05.466853 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a9a44-35a3-45be-93ab-1cd30255ddc1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:05.471653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5f1b3-370a-402c-b81c-5f969b4df76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:05.472317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5f1b3-370a-402c-b81c-5f969b4df76a admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.472608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5f1b3-370a-402c-b81c-5f969b4df76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.472881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5f1b3-370a-402c-b81c-5f969b4df76a admin admin] PATH_INFO: `/v3/projects/4c0c70b33d84477a830ea4e277424b72/users/6394c0788255419f9f9b0bd70b308680/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.487693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5f1b3-370a-402c-b81c-5f969b4df76a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c0c70b33d84477a830ea4e277424b72, user_id=6394c0788255419f9f9b0bd70b308680, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:05.490886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5f1b3-370a-402c-b81c-5f969b4df76a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:05.507423 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1130/2268] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:17:05 2022] PUT /identity/v3/projects/4c0c70b33d84477a830ea4e277424b72/users/6394c0788255419f9f9b0bd70b308680/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:05.519518 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421b381-2ffb-4369-8678-a3ca6d9a95d8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.520095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421b381-2ffb-4369-8678-a3ca6d9a95d8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.520095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d421b381-2ffb-4369-8678-a3ca6d9a95d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.558782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d421b381-2ffb-4369-8678-a3ca6d9a95d8 None None] MFA Rules not processed for user `6394c0788255419f9f9b0bd70b308680`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:05.560016 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d421b381-2ffb-4369-8678-a3ca6d9a95d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:05.632853 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1139/2269] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:17:05.639253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5f1b3-370a-402c-b81c-5f969b4df76a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:05.690119 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:05.714953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5f1b3-370a-402c-b81c-5f969b4df76a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:05.719641 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e0dd9-8984-425f-9f50-f535441ac6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:05.720263 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e0dd9-8984-425f-9f50-f535441ac6f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.720552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e0dd9-8984-425f-9f50-f535441ac6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.720830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e0dd9-8984-425f-9f50-f535441ac6f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.721658 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e0dd9-8984-425f-9f50-f535441ac6f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:05.724155 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e0dd9-8984-425f-9f50-f535441ac6f0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:05.755647 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1131/2270] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:17:05 2022] POST /identity/v3/projects => generated 392 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:17:05.756887 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421b381-2ffb-4369-8678-a3ca6d9a95d8 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:05.767633 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d421b381-2ffb-4369-8678-a3ca6d9a95d8 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:05.831220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247765c-925e-4669-babb-3b1ba9ca8116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:05.831952 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247765c-925e-4669-babb-3b1ba9ca8116 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.832382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247765c-925e-4669-babb-3b1ba9ca8116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.832769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247765c-925e-4669-babb-3b1ba9ca8116 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.833540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247765c-925e-4669-babb-3b1ba9ca8116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:05.835599 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247765c-925e-4669-babb-3b1ba9ca8116 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:05.859310 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1140/2271] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:17:05 2022] POST /identity/v3/users => generated 303 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:17:05.863219 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e0dd9-8984-425f-9f50-f535441ac6f0 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:05.874863 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e0dd9-8984-425f-9f50-f535441ac6f0 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:05.885638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab915a3-871c-4941-98ac-d4533de0d921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:05.886397 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab915a3-871c-4941-98ac-d4533de0d921 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.886857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab915a3-871c-4941-98ac-d4533de0d921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.887263 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab915a3-871c-4941-98ac-d4533de0d921 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.888154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab915a3-871c-4941-98ac-d4533de0d921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:05.890647 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab915a3-871c-4941-98ac-d4533de0d921 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:05.911164 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1132/2272] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:05 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:17:05.912773 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247765c-925e-4669-babb-3b1ba9ca8116 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:05.934495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247765c-925e-4669-babb-3b1ba9ca8116 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:05.939310 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329879d-c533-4044-b854-eba5decbbde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:05.939976 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329879d-c533-4044-b854-eba5decbbde2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:05.940256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329879d-c533-4044-b854-eba5decbbde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:05.940531 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329879d-c533-4044-b854-eba5decbbde2 admin admin] PATH_INFO: `/v3/projects/1cd3cfaab3c84748a15145dd15c937a9/users/61a7ef1c8bc543ef850be737bee1c161/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:05.956317 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329879d-c533-4044-b854-eba5decbbde2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd3cfaab3c84748a15145dd15c937a9, user_id=61a7ef1c8bc543ef850be737bee1c161, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:05.959201 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329879d-c533-4044-b854-eba5decbbde2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:05.972699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1141/2273] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:17:05 2022] PUT /identity/v3/projects/1cd3cfaab3c84748a15145dd15c937a9/users/61a7ef1c8bc543ef850be737bee1c161/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:05.976984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab915a3-871c-4941-98ac-d4533de0d921 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:05.988006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab915a3-871c-4941-98ac-d4533de0d921 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:06.079460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14022854-b1fe-4d31-b728-1a17350da403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:06.081211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14022854-b1fe-4d31-b728-1a17350da403 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:06.081695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14022854-b1fe-4d31-b728-1a17350da403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:06.082963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14022854-b1fe-4d31-b728-1a17350da403 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:06.084269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14022854-b1fe-4d31-b728-1a17350da403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:06.087165 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14022854-b1fe-4d31-b728-1a17350da403 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:06.107250 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1133/2274] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:05 2022] GET /identity/v3/roles => generated 1427 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:17:06.111440 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329879d-c533-4044-b854-eba5decbbde2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:06.134906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329879d-c533-4044-b854-eba5decbbde2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:06.134906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbce36-20ab-4186-ae43-908c6813da36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:06.134906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbce36-20ab-4186-ae43-908c6813da36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:06.134906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbce36-20ab-4186-ae43-908c6813da36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:06.135300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffbce36-20ab-4186-ae43-908c6813da36 admin admin] PATH_INFO: `/v3/projects/1cd3cfaab3c84748a15145dd15c937a9/users/61a7ef1c8bc543ef850be737bee1c161/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:06.135300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffbce36-20ab-4186-ae43-908c6813da36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd3cfaab3c84748a15145dd15c937a9, user_id=61a7ef1c8bc543ef850be737bee1c161, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:06.135300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffbce36-20ab-4186-ae43-908c6813da36 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:06.146942 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1142/2275] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:17:06 2022] PUT /identity/v3/projects/1cd3cfaab3c84748a15145dd15c937a9/users/61a7ef1c8bc543ef850be737bee1c161/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:17:06.157525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0c74c-6aa0-4e5b-bad8-3bbbf93a8640 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:06.157821 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0c74c-6aa0-4e5b-bad8-3bbbf93a8640 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:06.158109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0c74c-6aa0-4e5b-bad8-3bbbf93a8640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:06.197839 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e8e0c74c-6aa0-4e5b-bad8-3bbbf93a8640 None None] MFA Rules not processed for user `61a7ef1c8bc543ef850be737bee1c161`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:06.199532 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e8e0c74c-6aa0-4e5b-bad8-3bbbf93a8640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:06.282908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1134/2276] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:06 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:06.294434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbce36-20ab-4186-ae43-908c6813da36 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:06.296215 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-0ffbce36-20ab-4186-ae43-908c6813da36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:06.316952 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:06.359682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffbce36-20ab-4186-ae43-908c6813da36 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:06.363472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f727892-085f-4d55-a41b-cfd3ea9e38a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:06.364123 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f727892-085f-4d55-a41b-cfd3ea9e38a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:06.364407 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f727892-085f-4d55-a41b-cfd3ea9e38a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:06.364684 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f727892-085f-4d55-a41b-cfd3ea9e38a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:06.375348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f727892-085f-4d55-a41b-cfd3ea9e38a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:06.376916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f727892-085f-4d55-a41b-cfd3ea9e38a4 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:06.387023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1143/2277] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:17:06 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:06.437984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0c74c-6aa0-4e5b-bad8-3bbbf93a8640 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:06.439474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e8e0c74c-6aa0-4e5b-bad8-3bbbf93a8640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:06.452126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0c74c-6aa0-4e5b-bad8-3bbbf93a8640 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:06.456047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13623ed-cd4a-443f-bd33-616ca0032c48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:06.456670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13623ed-cd4a-443f-bd33-616ca0032c48 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:06.456956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13623ed-cd4a-443f-bd33-616ca0032c48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:06.457237 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13623ed-cd4a-443f-bd33-616ca0032c48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:06.492939 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:06.519665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13623ed-cd4a-443f-bd33-616ca0032c48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:06.521712 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13623ed-cd4a-443f-bd33-616ca0032c48 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:06.533029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1135/2278] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:17:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:11.044972 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f727892-085f-4d55-a41b-cfd3ea9e38a4 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:11.119341 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f727892-085f-4d55-a41b-cfd3ea9e38a4 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:11.124009 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06354cc-991a-4b3a-b889-a2c24f267170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:11.124654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06354cc-991a-4b3a-b889-a2c24f267170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:11.124935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06354cc-991a-4b3a-b889-a2c24f267170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:11.125233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06354cc-991a-4b3a-b889-a2c24f267170 admin admin] PATH_INFO: `/v3/users/6394c0788255419f9f9b0bd70b308680` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:11.126415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06354cc-991a-4b3a-b889-a2c24f267170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6394c0788255419f9f9b0bd70b308680)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:11.128519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06354cc-991a-4b3a-b889-a2c24f267170 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:11.185391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a06354cc-991a-4b3a-b889-a2c24f267170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6394c0788255419f9f9b0bd70b308680'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:11.186111 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a06354cc-991a-4b3a-b889-a2c24f267170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6394c0788255419f9f9b0bd70b308680'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:11.194888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a06354cc-991a-4b3a-b889-a2c24f267170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6394c0788255419f9f9b0bd70b308680'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:11.195649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a06354cc-991a-4b3a-b889-a2c24f267170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6394c0788255419f9f9b0bd70b308680'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:11.209122 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a06354cc-991a-4b3a-b889-a2c24f267170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6394c0788255419f9f9b0bd70b308680'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:11.223260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1144/2279] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:17:11 2022] DELETE /identity/v3/users/6394c0788255419f9f9b0bd70b308680 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:11.227745 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147cb40-7933-492e-819d-53140326fb21 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:11.228241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147cb40-7933-492e-819d-53140326fb21 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:11.228654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147cb40-7933-492e-819d-53140326fb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:11.248575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d147cb40-7933-492e-819d-53140326fb21 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:11.249945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d147cb40-7933-492e-819d-53140326fb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:11.314590 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1136/2280] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:11.324963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06354cc-991a-4b3a-b889-a2c24f267170 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:11.327569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a06354cc-991a-4b3a-b889-a2c24f267170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:11.373880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:11.393337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06354cc-991a-4b3a-b889-a2c24f267170 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:11.397296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834592f3-ce94-4e2b-a3a3-6508950395bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:11.397923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834592f3-ce94-4e2b-a3a3-6508950395bf service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:11.398209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834592f3-ce94-4e2b-a3a3-6508950395bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:11.398491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834592f3-ce94-4e2b-a3a3-6508950395bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:11.408893 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834592f3-ce94-4e2b-a3a3-6508950395bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:11.410417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834592f3-ce94-4e2b-a3a3-6508950395bf service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:11.423072 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1145/2281] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:17:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:11.789500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147cb40-7933-492e-819d-53140326fb21 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:11.791050 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d147cb40-7933-492e-819d-53140326fb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:11.808343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147cb40-7933-492e-819d-53140326fb21 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:11.813128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:11.813785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:11.814060 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:11.814926 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] PATH_INFO: `/v3/projects/4c0c70b33d84477a830ea4e277424b72` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:11.815809 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c0c70b33d84477a830ea4e277424b72)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:11.817726 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:11.882630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c0c70b33d84477a830ea4e277424b72', 'request_id': 'req-d8357464-6255-4958-b927-c0f0ec9066db'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:11.891695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c0c70b33d84477a830ea4e277424b72', 'request_id': 'req-d8357464-6255-4958-b927-c0f0ec9066db'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:11.904110 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c0c70b33d84477a830ea4e277424b72', 'request_id': 'req-d8357464-6255-4958-b927-c0f0ec9066db'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:11.921743 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] The token cache is being invalidate because project 4c0c70b33d84477a830ea4e277424b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:17:11.922125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:11.923310 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:11.925427 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1137/2282] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:17:11 2022] DELETE /identity/v3/projects/4c0c70b33d84477a830ea4e277424b72 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:11.930352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834592f3-ce94-4e2b-a3a3-6508950395bf service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:11.942208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834592f3-ce94-4e2b-a3a3-6508950395bf service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:12.013819 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2ef99-456e-46ff-98b2-a9d320d6cdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:12.014508 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2ef99-456e-46ff-98b2-a9d320d6cdf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:12.014844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2ef99-456e-46ff-98b2-a9d320d6cdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:12.015091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2ef99-456e-46ff-98b2-a9d320d6cdf6 admin admin] PATH_INFO: `/v3/users/61a7ef1c8bc543ef850be737bee1c161` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:12.016277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2ef99-456e-46ff-98b2-a9d320d6cdf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a7ef1c8bc543ef850be737bee1c161)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:12.018250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2ef99-456e-46ff-98b2-a9d320d6cdf6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:12.072837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21d2ef99-456e-46ff-98b2-a9d320d6cdf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a7ef1c8bc543ef850be737bee1c161'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:12.073647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21d2ef99-456e-46ff-98b2-a9d320d6cdf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a7ef1c8bc543ef850be737bee1c161'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:12.084031 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21d2ef99-456e-46ff-98b2-a9d320d6cdf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a7ef1c8bc543ef850be737bee1c161'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:12.084740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21d2ef99-456e-46ff-98b2-a9d320d6cdf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a7ef1c8bc543ef850be737bee1c161'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:12.097825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-21d2ef99-456e-46ff-98b2-a9d320d6cdf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a7ef1c8bc543ef850be737bee1c161'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:12.112820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1146/2283] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:17:11 2022] DELETE /identity/v3/users/61a7ef1c8bc543ef850be737bee1c161 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:12.712288 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:12.714338 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:12.731540 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8357464-6255-4958-b927-c0f0ec9066db admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:12.811566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc81d2-8d3a-4c1a-9069-6163893624e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:12.812322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc81d2-8d3a-4c1a-9069-6163893624e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:12.812646 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc81d2-8d3a-4c1a-9069-6163893624e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:12.813333 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc81d2-8d3a-4c1a-9069-6163893624e0 admin admin] PATH_INFO: `/v3/projects/1cd3cfaab3c84748a15145dd15c937a9` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:12.815977 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc81d2-8d3a-4c1a-9069-6163893624e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd3cfaab3c84748a15145dd15c937a9)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:12.818039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfc81d2-8d3a-4c1a-9069-6163893624e0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:12.819872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:12.882317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ddfc81d2-8d3a-4c1a-9069-6163893624e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd3cfaab3c84748a15145dd15c937a9', 'request_id': 'req-ddfc81d2-8d3a-4c1a-9069-6163893624e0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:12.889391 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ddfc81d2-8d3a-4c1a-9069-6163893624e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd3cfaab3c84748a15145dd15c937a9', 'request_id': 'req-ddfc81d2-8d3a-4c1a-9069-6163893624e0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:12.903026 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ddfc81d2-8d3a-4c1a-9069-6163893624e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd3cfaab3c84748a15145dd15c937a9', 'request_id': 'req-ddfc81d2-8d3a-4c1a-9069-6163893624e0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:12.917583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ddfc81d2-8d3a-4c1a-9069-6163893624e0 admin admin] The token cache is being invalidate because project 1cd3cfaab3c84748a15145dd15c937a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:17:12.917749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ddfc81d2-8d3a-4c1a-9069-6163893624e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:12.918694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ddfc81d2-8d3a-4c1a-9069-6163893624e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:12.920368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1138/2284] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:17:12 2022] DELETE /identity/v3/projects/1cd3cfaab3c84748a15145dd15c937a9 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:12.925233 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a240a-aa7c-4510-a013-f2bd01fc78ca None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:12.925524 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a240a-aa7c-4510-a013-f2bd01fc78ca None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:12.925848 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18a240a-aa7c-4510-a013-f2bd01fc78ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:12.949550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c18a240a-aa7c-4510-a013-f2bd01fc78ca None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:12.951141 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c18a240a-aa7c-4510-a013-f2bd01fc78ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:13.029483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1147/2285] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:13.041412 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e5845-e9d2-4fcd-b583-e1742c510c6f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.042289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e5845-e9d2-4fcd-b583-e1742c510c6f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.042848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e5845-e9d2-4fcd-b583-e1742c510c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.064898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d67e5845-e9d2-4fcd-b583-e1742c510c6f None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:13.068124 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d67e5845-e9d2-4fcd-b583-e1742c510c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:13.083074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1139/2286] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:13.085342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a240a-aa7c-4510-a013-f2bd01fc78ca None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:13.091102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:13.099874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18a240a-aa7c-4510-a013-f2bd01fc78ca None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:13.109058 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ca23c-57dc-4ae9-8149-0aa9efb627d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:13.110855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ca23c-57dc-4ae9-8149-0aa9efb627d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.112247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ca23c-57dc-4ae9-8149-0aa9efb627d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.112548 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ca23c-57dc-4ae9-8149-0aa9efb627d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.115660 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ca23c-57dc-4ae9-8149-0aa9efb627d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:17:13.116531 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ca23c-57dc-4ae9-8149-0aa9efb627d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:13.121444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ca23c-57dc-4ae9-8149-0aa9efb627d8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:13.152749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1148/2287] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:17:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:17:13.154265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e5845-e9d2-4fcd-b583-e1742c510c6f None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:13.168545 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e5845-e9d2-4fcd-b583-e1742c510c6f None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:13.176774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3da2b-7990-40e1-9158-d9c0f495989d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:13.179218 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3da2b-7990-40e1-9158-d9c0f495989d admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.179641 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3da2b-7990-40e1-9158-d9c0f495989d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.180132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3da2b-7990-40e1-9158-d9c0f495989d admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.181597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3da2b-7990-40e1-9158-d9c0f495989d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:13.184235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3da2b-7990-40e1-9158-d9c0f495989d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:13.222108 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1140/2288] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:17:13 2022] POST /identity/v3/projects => generated 376 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:17:13.227019 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ca23c-57dc-4ae9-8149-0aa9efb627d8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:13.237529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ca23c-57dc-4ae9-8149-0aa9efb627d8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:13.304723 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b49e35-549e-41db-b557-22fa0d7e8e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:13.305372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b49e35-549e-41db-b557-22fa0d7e8e00 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.305659 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b49e35-549e-41db-b557-22fa0d7e8e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.305945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b49e35-549e-41db-b557-22fa0d7e8e00 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.306653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b49e35-549e-41db-b557-22fa0d7e8e00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:13.308889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b49e35-549e-41db-b557-22fa0d7e8e00 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:13.336186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1149/2289] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:17:13 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:17:13.340496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3da2b-7990-40e1-9158-d9c0f495989d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:13.352410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3da2b-7990-40e1-9158-d9c0f495989d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:13.357365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee11f95-2676-46e4-9414-8bb8ddd0341e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:13.358006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee11f95-2676-46e4-9414-8bb8ddd0341e admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.358292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee11f95-2676-46e4-9414-8bb8ddd0341e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.358588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee11f95-2676-46e4-9414-8bb8ddd0341e admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.359329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee11f95-2676-46e4-9414-8bb8ddd0341e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:13.361007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee11f95-2676-46e4-9414-8bb8ddd0341e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:13.372534 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1141/2290] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:13 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:17:13.376785 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b49e35-549e-41db-b557-22fa0d7e8e00 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:13.394256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b49e35-549e-41db-b557-22fa0d7e8e00 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:13.402204 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bc342-01b4-4eec-81fc-35af21e00edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:13.402873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bc342-01b4-4eec-81fc-35af21e00edc admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.403161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bc342-01b4-4eec-81fc-35af21e00edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.403442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bc342-01b4-4eec-81fc-35af21e00edc admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.404157 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bc342-01b4-4eec-81fc-35af21e00edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:13.406217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bc342-01b4-4eec-81fc-35af21e00edc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:13.420782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1150/2291] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:13 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:17:13.425459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee11f95-2676-46e4-9414-8bb8ddd0341e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:13.436234 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee11f95-2676-46e4-9414-8bb8ddd0341e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:13.441271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10965fd6-d567-49af-9b92-a1f415ac2ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:13.441990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10965fd6-d567-49af-9b92-a1f415ac2ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.442277 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10965fd6-d567-49af-9b92-a1f415ac2ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.442557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10965fd6-d567-49af-9b92-a1f415ac2ea2 admin admin] PATH_INFO: `/v3/projects/747f7db717fb4c28b3039b4d2f161a54/users/3321bd1b18df43cdbe4fc8a0958d8c9f/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.461590 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10965fd6-d567-49af-9b92-a1f415ac2ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=747f7db717fb4c28b3039b4d2f161a54, user_id=3321bd1b18df43cdbe4fc8a0958d8c9f, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:13.464531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10965fd6-d567-49af-9b92-a1f415ac2ea2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:13.492261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1142/2292] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:17:13 2022] PUT /identity/v3/projects/747f7db717fb4c28b3039b4d2f161a54/users/3321bd1b18df43cdbe4fc8a0958d8c9f/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:17:13.496752 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fa7a6-0582-420b-9388-5224697af9ab None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.496752 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fa7a6-0582-420b-9388-5224697af9ab None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.497604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fa7a6-0582-420b-9388-5224697af9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.557176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e31fa7a6-0582-420b-9388-5224697af9ab None None] MFA Rules not processed for user `3321bd1b18df43cdbe4fc8a0958d8c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:13.558439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e31fa7a6-0582-420b-9388-5224697af9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:13.632672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1151/2293] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:13.637202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10965fd6-d567-49af-9b92-a1f415ac2ea2 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:13.709848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10965fd6-d567-49af-9b92-a1f415ac2ea2 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:13.715270 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbefab-9be5-4bb4-af5f-93d104d91df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:13.715996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbefab-9be5-4bb4-af5f-93d104d91df3 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.716425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbefab-9be5-4bb4-af5f-93d104d91df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.716867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbefab-9be5-4bb4-af5f-93d104d91df3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.717988 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbefab-9be5-4bb4-af5f-93d104d91df3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:13.720305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbefab-9be5-4bb4-af5f-93d104d91df3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:13.763456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1143/2294] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:17:13 2022] POST /identity/v3/projects => generated 376 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:17:13.764714 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fa7a6-0582-420b-9388-5224697af9ab None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:13.777465 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fa7a6-0582-420b-9388-5224697af9ab None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:13.849591 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:13.856920 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813643d5-a939-4972-ba85-94a86dfc43fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:13.857944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813643d5-a939-4972-ba85-94a86dfc43fc admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.858515 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813643d5-a939-4972-ba85-94a86dfc43fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.859135 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813643d5-a939-4972-ba85-94a86dfc43fc admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.860195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813643d5-a939-4972-ba85-94a86dfc43fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:13.862262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813643d5-a939-4972-ba85-94a86dfc43fc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:13.890386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1152/2295] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:17:13 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:17:13.894710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbefab-9be5-4bb4-af5f-93d104d91df3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:13.905633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbefab-9be5-4bb4-af5f-93d104d91df3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:13.910418 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bee976-9c9a-41e8-a6f6-b7afbc1e2d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:13.911073 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bee976-9c9a-41e8-a6f6-b7afbc1e2d6c admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.911355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bee976-9c9a-41e8-a6f6-b7afbc1e2d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.911627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bee976-9c9a-41e8-a6f6-b7afbc1e2d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.912335 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bee976-9c9a-41e8-a6f6-b7afbc1e2d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:13.913949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bee976-9c9a-41e8-a6f6-b7afbc1e2d6c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:13.919743 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:13.929743 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1144/2296] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:13 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:17:13.930994 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813643d5-a939-4972-ba85-94a86dfc43fc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:13.942234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813643d5-a939-4972-ba85-94a86dfc43fc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:13.947461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c8198-abfe-43d7-b175-b519274465fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:13.948160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c8198-abfe-43d7-b175-b519274465fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:13.948473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c8198-abfe-43d7-b175-b519274465fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:13.948781 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c8198-abfe-43d7-b175-b519274465fd admin admin] PATH_INFO: `/v3/projects/b7d2373aae244d2db9d474b169508a7c/users/0954b2de977c4af8810113dae4ad67d6/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:13.965968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c8198-abfe-43d7-b175-b519274465fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d2373aae244d2db9d474b169508a7c, user_id=0954b2de977c4af8810113dae4ad67d6, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:13.968856 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c8198-abfe-43d7-b175-b519274465fd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:13.983405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1153/2297] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:17:13 2022] PUT /identity/v3/projects/b7d2373aae244d2db9d474b169508a7c/users/0954b2de977c4af8810113dae4ad67d6/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:17:13.988660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bee976-9c9a-41e8-a6f6-b7afbc1e2d6c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:13.999134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bee976-9c9a-41e8-a6f6-b7afbc1e2d6c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:14.063031 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f19aa-bb4c-4f4b-9615-d3180cc58226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:14.063681 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f19aa-bb4c-4f4b-9615-d3180cc58226 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:14.063954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f19aa-bb4c-4f4b-9615-d3180cc58226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:14.064243 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f19aa-bb4c-4f4b-9615-d3180cc58226 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:14.065056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f19aa-bb4c-4f4b-9615-d3180cc58226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:14.067269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f19aa-bb4c-4f4b-9615-d3180cc58226 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:14.078053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1145/2298] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:13 2022] GET /identity/v3/roles => generated 1427 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:17:14.082630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c8198-abfe-43d7-b175-b519274465fd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:14.094716 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c8198-abfe-43d7-b175-b519274465fd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:14.099548 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741905b9-efe9-4446-8ec0-70facb43b59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:14.100217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741905b9-efe9-4446-8ec0-70facb43b59f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:14.100512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741905b9-efe9-4446-8ec0-70facb43b59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:14.100788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741905b9-efe9-4446-8ec0-70facb43b59f admin admin] PATH_INFO: `/v3/projects/b7d2373aae244d2db9d474b169508a7c/users/0954b2de977c4af8810113dae4ad67d6/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:14.103062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741905b9-efe9-4446-8ec0-70facb43b59f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d2373aae244d2db9d474b169508a7c, user_id=0954b2de977c4af8810113dae4ad67d6, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:14.105724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741905b9-efe9-4446-8ec0-70facb43b59f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:14.117433 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1154/2299] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:17:14 2022] PUT /identity/v3/projects/b7d2373aae244d2db9d474b169508a7c/users/0954b2de977c4af8810113dae4ad67d6/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:14.127670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f48683-6453-40ee-8ab3-850e96912607 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:14.127962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f48683-6453-40ee-8ab3-850e96912607 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:14.128242 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f48683-6453-40ee-8ab3-850e96912607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:14.159251 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-72f48683-6453-40ee-8ab3-850e96912607 None None] MFA Rules not processed for user `0954b2de977c4af8810113dae4ad67d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:14.160784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-72f48683-6453-40ee-8ab3-850e96912607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:14.236582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1146/2300] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:14.248926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741905b9-efe9-4446-8ec0-70facb43b59f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:14.250971 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-741905b9-efe9-4446-8ec0-70facb43b59f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:14.320173 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741905b9-efe9-4446-8ec0-70facb43b59f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:14.324073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9d969-53f6-4076-9796-3431622ea4f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:14.324694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9d969-53f6-4076-9796-3431622ea4f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:14.324974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9d969-53f6-4076-9796-3431622ea4f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:14.325250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9d969-53f6-4076-9796-3431622ea4f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:14.383145 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9d969-53f6-4076-9796-3431622ea4f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:14.384600 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9d969-53f6-4076-9796-3431622ea4f1 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:14.394038 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1155/2301] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:17:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:18.084194 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f48683-6453-40ee-8ab3-850e96912607 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:18.163283 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f48683-6453-40ee-8ab3-850e96912607 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:18.167711 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c555f-a8a1-4aa0-aa9a-c3f10007a18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:18.168873 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c555f-a8a1-4aa0-aa9a-c3f10007a18f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:18.169148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c555f-a8a1-4aa0-aa9a-c3f10007a18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:18.169421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c555f-a8a1-4aa0-aa9a-c3f10007a18f admin admin] PATH_INFO: `/v3/users/3321bd1b18df43cdbe4fc8a0958d8c9f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:18.170679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c555f-a8a1-4aa0-aa9a-c3f10007a18f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3321bd1b18df43cdbe4fc8a0958d8c9f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:18.173040 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c555f-a8a1-4aa0-aa9a-c3f10007a18f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:18.212782 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:18.250082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d47c555f-a8a1-4aa0-aa9a-c3f10007a18f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3321bd1b18df43cdbe4fc8a0958d8c9f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:18.264386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d47c555f-a8a1-4aa0-aa9a-c3f10007a18f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3321bd1b18df43cdbe4fc8a0958d8c9f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:18.265036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d47c555f-a8a1-4aa0-aa9a-c3f10007a18f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3321bd1b18df43cdbe4fc8a0958d8c9f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:18.267449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d47c555f-a8a1-4aa0-aa9a-c3f10007a18f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3321bd1b18df43cdbe4fc8a0958d8c9f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:18.296000 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d47c555f-a8a1-4aa0-aa9a-c3f10007a18f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3321bd1b18df43cdbe4fc8a0958d8c9f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:18.310394 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1147/2302] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:17:18 2022] DELETE /identity/v3/users/3321bd1b18df43cdbe4fc8a0958d8c9f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:18.314891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7d232-1ab6-43be-8520-c39605846c74 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:18.315403 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7d232-1ab6-43be-8520-c39605846c74 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:18.315523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7d232-1ab6-43be-8520-c39605846c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:18.334742 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-41a7d232-1ab6-43be-8520-c39605846c74 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:18.335978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-41a7d232-1ab6-43be-8520-c39605846c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:18.408192 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1156/2303] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:18.420769 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c555f-a8a1-4aa0-aa9a-c3f10007a18f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:18.422422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d47c555f-a8a1-4aa0-aa9a-c3f10007a18f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:18.491326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c555f-a8a1-4aa0-aa9a-c3f10007a18f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:18.495082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae42b2c-34f9-4934-82ea-1a9f75acbd4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:18.495716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae42b2c-34f9-4934-82ea-1a9f75acbd4d service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:18.496006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae42b2c-34f9-4934-82ea-1a9f75acbd4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:18.496279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae42b2c-34f9-4934-82ea-1a9f75acbd4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:18.507670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae42b2c-34f9-4934-82ea-1a9f75acbd4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:18.509120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae42b2c-34f9-4934-82ea-1a9f75acbd4d service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:18.520412 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1148/2304] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:18.844095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7d232-1ab6-43be-8520-c39605846c74 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:18.845560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-41a7d232-1ab6-43be-8520-c39605846c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:18.856366 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:18.867191 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7d232-1ab6-43be-8520-c39605846c74 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:18.872968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:18.874172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:18.877314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:18.877314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] PATH_INFO: `/v3/projects/747f7db717fb4c28b3039b4d2f161a54` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:18.877788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=747f7db717fb4c28b3039b4d2f161a54)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:18.880079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:18.931965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '747f7db717fb4c28b3039b4d2f161a54', 'request_id': 'req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:18.942823 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '747f7db717fb4c28b3039b4d2f161a54', 'request_id': 'req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:18.953449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '747f7db717fb4c28b3039b4d2f161a54', 'request_id': 'req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:18.976331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] The token cache is being invalidate because project 747f7db717fb4c28b3039b4d2f161a54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:17:18.976760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:18.978628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:18.980277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1157/2305] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:17:18 2022] DELETE /identity/v3/projects/747f7db717fb4c28b3039b4d2f161a54 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:17:18.984224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae42b2c-34f9-4934-82ea-1a9f75acbd4d service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:18.994127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae42b2c-34f9-4934-82ea-1a9f75acbd4d service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:19.051462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e5aa4-3160-4ab2-9716-6ff9b4eba098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:19.052224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e5aa4-3160-4ab2-9716-6ff9b4eba098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:19.052512 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e5aa4-3160-4ab2-9716-6ff9b4eba098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:19.052792 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e5aa4-3160-4ab2-9716-6ff9b4eba098 admin admin] PATH_INFO: `/v3/users/0954b2de977c4af8810113dae4ad67d6` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:19.053977 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e5aa4-3160-4ab2-9716-6ff9b4eba098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0954b2de977c4af8810113dae4ad67d6)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:19.055945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e5aa4-3160-4ab2-9716-6ff9b4eba098 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:19.103961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:19.105708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-471e5aa4-3160-4ab2-9716-6ff9b4eba098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0954b2de977c4af8810113dae4ad67d6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:19.113085 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-471e5aa4-3160-4ab2-9716-6ff9b4eba098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0954b2de977c4af8810113dae4ad67d6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:19.114074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-471e5aa4-3160-4ab2-9716-6ff9b4eba098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0954b2de977c4af8810113dae4ad67d6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:19.114901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-471e5aa4-3160-4ab2-9716-6ff9b4eba098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0954b2de977c4af8810113dae4ad67d6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:19.127107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-471e5aa4-3160-4ab2-9716-6ff9b4eba098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0954b2de977c4af8810113dae4ad67d6'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:19.140383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1149/2306] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:17:18 2022] DELETE /identity/v3/users/0954b2de977c4af8810113dae4ad67d6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:19.433685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:19.436800 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:19.457816 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af3c5a-dfa9-4b6c-8a78-282aa33ee6b3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:19.527648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:19.528306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:19.528594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:19.528873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a admin admin] PATH_INFO: `/v3/projects/b7d2373aae244d2db9d474b169508a7c` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:19.530154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d2373aae244d2db9d474b169508a7c)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:19.532209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:19.587093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d2373aae244d2db9d474b169508a7c', 'request_id': 'req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:19.594213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d2373aae244d2db9d474b169508a7c', 'request_id': 'req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:19.600264 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d2373aae244d2db9d474b169508a7c', 'request_id': 'req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:19.614595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a admin admin] The token cache is being invalidate because project b7d2373aae244d2db9d474b169508a7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:17:19.614956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:19.615680 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d7077dd4-5413-43b7-8b6a-0b13faf4e89a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:19.617472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1158/2307] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:17:19 2022] DELETE /identity/v3/projects/b7d2373aae244d2db9d474b169508a7c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:19.625262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb92aa1-94dc-41f6-80e5-7fcbc7c0cfba None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:19.625660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb92aa1-94dc-41f6-80e5-7fcbc7c0cfba None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:19.626016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb92aa1-94dc-41f6-80e5-7fcbc7c0cfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:19.651188 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-bbb92aa1-94dc-41f6-80e5-7fcbc7c0cfba None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:19.652522 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bbb92aa1-94dc-41f6-80e5-7fcbc7c0cfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:19.736155 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1150/2308] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:19.747980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b690056-d9f6-474e-8040-05a0846c7356 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:19.748276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b690056-d9f6-474e-8040-05a0846c7356 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:19.748559 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b690056-d9f6-474e-8040-05a0846c7356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:19.771944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6b690056-d9f6-474e-8040-05a0846c7356 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:19.771944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6b690056-d9f6-474e-8040-05a0846c7356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:19.780007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1159/2309] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:19.784281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb92aa1-94dc-41f6-80e5-7fcbc7c0cfba None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:19.799661 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb92aa1-94dc-41f6-80e5-7fcbc7c0cfba None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:19.805130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ea991-a44f-42bd-ae78-033d385455db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:19.805844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ea991-a44f-42bd-ae78-033d385455db admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:19.806133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ea991-a44f-42bd-ae78-033d385455db admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:19.806401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ea991-a44f-42bd-ae78-033d385455db admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:19.807156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ea991-a44f-42bd-ae78-033d385455db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:17:19.807469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ea991-a44f-42bd-ae78-033d385455db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:19.809097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0ea991-a44f-42bd-ae78-033d385455db admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:19.833574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1151/2310] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:17:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:17:19.837883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b690056-d9f6-474e-8040-05a0846c7356 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:19.848059 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b690056-d9f6-474e-8040-05a0846c7356 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:19.853165 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64904b17-cdee-4445-9fa1-2554e9bd5972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:19.853685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64904b17-cdee-4445-9fa1-2554e9bd5972 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:19.853970 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64904b17-cdee-4445-9fa1-2554e9bd5972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:19.854247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64904b17-cdee-4445-9fa1-2554e9bd5972 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:19.854967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64904b17-cdee-4445-9fa1-2554e9bd5972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:19.856917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64904b17-cdee-4445-9fa1-2554e9bd5972 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:19.867577 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:19.891509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1160/2311] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:17:19 2022] POST /identity/v3/projects => generated 384 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:17:19.892640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ea991-a44f-42bd-ae78-033d385455db admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:19.903062 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0ea991-a44f-42bd-ae78-033d385455db admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:19.962303 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95994db3-a948-4eff-b818-5fba1ed9425a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:19.962966 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95994db3-a948-4eff-b818-5fba1ed9425a admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:19.963252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95994db3-a948-4eff-b818-5fba1ed9425a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:19.963531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95994db3-a948-4eff-b818-5fba1ed9425a admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:19.964215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95994db3-a948-4eff-b818-5fba1ed9425a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:19.966178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95994db3-a948-4eff-b818-5fba1ed9425a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:19.999332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1152/2312] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:17:19 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:17:20.000561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64904b17-cdee-4445-9fa1-2554e9bd5972 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:20.020521 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64904b17-cdee-4445-9fa1-2554e9bd5972 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:20.028010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e9116-d4b1-4395-9e9c-74135569921b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:20.028902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e9116-d4b1-4395-9e9c-74135569921b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:20.029245 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e9116-d4b1-4395-9e9c-74135569921b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:20.030147 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086e9116-d4b1-4395-9e9c-74135569921b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:20.030988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e9116-d4b1-4395-9e9c-74135569921b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:20.033308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086e9116-d4b1-4395-9e9c-74135569921b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:20.051993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1161/2313] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:19 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:17:20.057295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95994db3-a948-4eff-b818-5fba1ed9425a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:20.069763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95994db3-a948-4eff-b818-5fba1ed9425a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:20.075218 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d57b23f-9f6a-4745-834d-ea5b71063342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:20.075951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57b23f-9f6a-4745-834d-ea5b71063342 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:20.076446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57b23f-9f6a-4745-834d-ea5b71063342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:20.077873 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d57b23f-9f6a-4745-834d-ea5b71063342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:20.078602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57b23f-9f6a-4745-834d-ea5b71063342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:20.080257 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d57b23f-9f6a-4745-834d-ea5b71063342 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:20.100473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1153/2314] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:20 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:17:20.101645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e9116-d4b1-4395-9e9c-74135569921b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:20.113583 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086e9116-d4b1-4395-9e9c-74135569921b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:20.118856 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba349db-aacb-4077-a5b0-194ee61e21bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:20.119396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba349db-aacb-4077-a5b0-194ee61e21bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:20.119684 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba349db-aacb-4077-a5b0-194ee61e21bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:20.119961 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba349db-aacb-4077-a5b0-194ee61e21bb admin admin] PATH_INFO: `/v3/projects/3d62512a132041a2be588d949ced06d0/users/4800a61ea4244630b92dc61d5d80d89c/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:20.144812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba349db-aacb-4077-a5b0-194ee61e21bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d62512a132041a2be588d949ced06d0, user_id=4800a61ea4244630b92dc61d5d80d89c, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:20.147785 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba349db-aacb-4077-a5b0-194ee61e21bb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:20.176211 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1162/2315] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:17:20 2022] PUT /identity/v3/projects/3d62512a132041a2be588d949ced06d0/users/4800a61ea4244630b92dc61d5d80d89c/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:20.177597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75b063-5a19-4bb2-9c1f-f3160f1d5917 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:20.178370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75b063-5a19-4bb2-9c1f-f3160f1d5917 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:20.178681 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75b063-5a19-4bb2-9c1f-f3160f1d5917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:20.209426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9d75b063-5a19-4bb2-9c1f-f3160f1d5917 None None] MFA Rules not processed for user `4800a61ea4244630b92dc61d5d80d89c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:20.210830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9d75b063-5a19-4bb2-9c1f-f3160f1d5917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:20.231296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:20.286780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1154/2316] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:20.291360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba349db-aacb-4077-a5b0-194ee61e21bb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:20.357762 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba349db-aacb-4077-a5b0-194ee61e21bb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:20.365437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3077e-6b64-4f7a-965b-2939af364493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:20.366094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3077e-6b64-4f7a-965b-2939af364493 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:20.366373 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3077e-6b64-4f7a-965b-2939af364493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:20.366695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3077e-6b64-4f7a-965b-2939af364493 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:20.367423 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3077e-6b64-4f7a-965b-2939af364493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:20.369331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3077e-6b64-4f7a-965b-2939af364493 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:20.397013 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1163/2317] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:17:20 2022] POST /identity/v3/projects => generated 386 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:17:20.401274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75b063-5a19-4bb2-9c1f-f3160f1d5917 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:20.413780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75b063-5a19-4bb2-9c1f-f3160f1d5917 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:20.474626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8b803-e2e8-4d39-a68c-dd22f4d2fd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:20.475285 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8b803-e2e8-4d39-a68c-dd22f4d2fd45 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:20.475568 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8b803-e2e8-4d39-a68c-dd22f4d2fd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:20.475842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8b803-e2e8-4d39-a68c-dd22f4d2fd45 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:20.476583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8b803-e2e8-4d39-a68c-dd22f4d2fd45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:20.478618 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8b803-e2e8-4d39-a68c-dd22f4d2fd45 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:20.507514 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1155/2318] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:17:20 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:17:20.507788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3077e-6b64-4f7a-965b-2939af364493 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:20.517745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3077e-6b64-4f7a-965b-2939af364493 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:20.523048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792d760-29b9-4ba2-8a27-86d1a2fc99de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:20.523668 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d760-29b9-4ba2-8a27-86d1a2fc99de admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:20.523952 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d760-29b9-4ba2-8a27-86d1a2fc99de admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:20.524226 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792d760-29b9-4ba2-8a27-86d1a2fc99de admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:20.524922 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792d760-29b9-4ba2-8a27-86d1a2fc99de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:20.526511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792d760-29b9-4ba2-8a27-86d1a2fc99de admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:20.540343 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1164/2319] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:20 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:17:20.542780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8b803-e2e8-4d39-a68c-dd22f4d2fd45 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:20.554316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8b803-e2e8-4d39-a68c-dd22f4d2fd45 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:20.559007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272416a7-b24a-4a9e-8779-db0f0d61a6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:20.559649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272416a7-b24a-4a9e-8779-db0f0d61a6bc admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:20.559927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272416a7-b24a-4a9e-8779-db0f0d61a6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:20.560205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272416a7-b24a-4a9e-8779-db0f0d61a6bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:20.560900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272416a7-b24a-4a9e-8779-db0f0d61a6bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:20.562535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272416a7-b24a-4a9e-8779-db0f0d61a6bc admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:20.573815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1156/2320] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:20 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:17:20.578009 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792d760-29b9-4ba2-8a27-86d1a2fc99de admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:20.589562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792d760-29b9-4ba2-8a27-86d1a2fc99de admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:20.594502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043b396-c8c4-4ab7-a346-11c9114d71e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:20.595236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043b396-c8c4-4ab7-a346-11c9114d71e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:20.595522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043b396-c8c4-4ab7-a346-11c9114d71e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:20.595800 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d043b396-c8c4-4ab7-a346-11c9114d71e5 admin admin] PATH_INFO: `/v3/projects/81279e65218f440da6d6810a58c4c5dd/users/3f51428c41c94556855f850d047437bf/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:20.613321 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043b396-c8c4-4ab7-a346-11c9114d71e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81279e65218f440da6d6810a58c4c5dd, user_id=3f51428c41c94556855f850d047437bf, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:20.616006 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d043b396-c8c4-4ab7-a346-11c9114d71e5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:20.632582 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1165/2321] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:17:20 2022] PUT /identity/v3/projects/81279e65218f440da6d6810a58c4c5dd/users/3f51428c41c94556855f850d047437bf/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:17:20.643949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecf249-4c13-43ff-8dc4-26c3f1db902c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:20.644384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecf249-4c13-43ff-8dc4-26c3f1db902c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:20.645346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ecf249-4c13-43ff-8dc4-26c3f1db902c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:20.678814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d5ecf249-4c13-43ff-8dc4-26c3f1db902c None None] MFA Rules not processed for user `3f51428c41c94556855f850d047437bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:20.680079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d5ecf249-4c13-43ff-8dc4-26c3f1db902c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:20.759567 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1157/2322] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:20.763908 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043b396-c8c4-4ab7-a346-11c9114d71e5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:20.843244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d043b396-c8c4-4ab7-a346-11c9114d71e5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:20.843244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09e074-30b3-49d5-9c21-29a39e6954ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:20.843244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09e074-30b3-49d5-9c21-29a39e6954ca admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:20.843244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09e074-30b3-49d5-9c21-29a39e6954ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:20.843619 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09e074-30b3-49d5-9c21-29a39e6954ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:20.843619 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09e074-30b3-49d5-9c21-29a39e6954ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:20.845303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09e074-30b3-49d5-9c21-29a39e6954ca admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:20.858546 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:20.879876 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1166/2323] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:17:20 2022] POST /identity/v3/projects => generated 386 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:17:20.884007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecf249-4c13-43ff-8dc4-26c3f1db902c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:20.895486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ecf249-4c13-43ff-8dc4-26c3f1db902c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:20.920957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:20.969886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7914f0-4761-4ae7-b6eb-97d017ef556a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:20.970924 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7914f0-4761-4ae7-b6eb-97d017ef556a admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:20.971384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7914f0-4761-4ae7-b6eb-97d017ef556a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:20.971844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7914f0-4761-4ae7-b6eb-97d017ef556a admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:20.972978 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7914f0-4761-4ae7-b6eb-97d017ef556a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:20.976289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7914f0-4761-4ae7-b6eb-97d017ef556a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:21.005254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1158/2324] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:17:20 2022] POST /identity/v3/users => generated 300 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:17:21.006395 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09e074-30b3-49d5-9c21-29a39e6954ca admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:21.017575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09e074-30b3-49d5-9c21-29a39e6954ca admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:21.023280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d796050-defd-4901-833f-7a52186f3724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:21.023982 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d796050-defd-4901-833f-7a52186f3724 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:21.024208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d796050-defd-4901-833f-7a52186f3724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:21.024429 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d796050-defd-4901-833f-7a52186f3724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:21.025137 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d796050-defd-4901-833f-7a52186f3724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:21.026746 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d796050-defd-4901-833f-7a52186f3724 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:21.045746 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1167/2325] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:21 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:17:21.049633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7914f0-4761-4ae7-b6eb-97d017ef556a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:21.060812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7914f0-4761-4ae7-b6eb-97d017ef556a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:21.065776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e2f9e-b4fd-47de-8c6c-da11b914f8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:21.067042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e2f9e-b4fd-47de-8c6c-da11b914f8d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:21.067323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e2f9e-b4fd-47de-8c6c-da11b914f8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:21.069395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e2f9e-b4fd-47de-8c6c-da11b914f8d4 admin admin] PATH_INFO: `/v3/projects/791399b2833247ea96bb569dc648d831/users/15daec108538449184a1ac5fff6d8909/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:21.087492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e2f9e-b4fd-47de-8c6c-da11b914f8d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=791399b2833247ea96bb569dc648d831, user_id=15daec108538449184a1ac5fff6d8909, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:21.090298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e2f9e-b4fd-47de-8c6c-da11b914f8d4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:21.115803 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1159/2326] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:17:21 2022] PUT /identity/v3/projects/791399b2833247ea96bb569dc648d831/users/15daec108538449184a1ac5fff6d8909/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:21.120921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d796050-defd-4901-833f-7a52186f3724 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:21.132749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d796050-defd-4901-833f-7a52186f3724 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:21.233452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47477f0-2ef4-44e8-8b39-d1ff8c478e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:21.233452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47477f0-2ef4-44e8-8b39-d1ff8c478e3b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:21.233452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47477f0-2ef4-44e8-8b39-d1ff8c478e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:21.233452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47477f0-2ef4-44e8-8b39-d1ff8c478e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:21.233887 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47477f0-2ef4-44e8-8b39-d1ff8c478e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:21.233887 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47477f0-2ef4-44e8-8b39-d1ff8c478e3b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:21.245863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1168/2327] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:21 2022] GET /identity/v3/roles => generated 1427 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:17:21.250523 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e2f9e-b4fd-47de-8c6c-da11b914f8d4 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:21.263318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e2f9e-b4fd-47de-8c6c-da11b914f8d4 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:21.268459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15ffff-6768-4042-8406-f81a32ff6db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:21.269294 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15ffff-6768-4042-8406-f81a32ff6db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:21.269618 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15ffff-6768-4042-8406-f81a32ff6db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:21.269972 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15ffff-6768-4042-8406-f81a32ff6db5 admin admin] PATH_INFO: `/v3/projects/791399b2833247ea96bb569dc648d831/users/15daec108538449184a1ac5fff6d8909/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:21.272106 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15ffff-6768-4042-8406-f81a32ff6db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=791399b2833247ea96bb569dc648d831, user_id=15daec108538449184a1ac5fff6d8909, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:21.274993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15ffff-6768-4042-8406-f81a32ff6db5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:21.289555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1160/2328] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:17:21 2022] PUT /identity/v3/projects/791399b2833247ea96bb569dc648d831/users/15daec108538449184a1ac5fff6d8909/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:21.299385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31281b1-53b1-447f-89d4-956a48d16476 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:21.299750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31281b1-53b1-447f-89d4-956a48d16476 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:21.300101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31281b1-53b1-447f-89d4-956a48d16476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:21.330025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f31281b1-53b1-447f-89d4-956a48d16476 None None] MFA Rules not processed for user `15daec108538449184a1ac5fff6d8909`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:21.331594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f31281b1-53b1-447f-89d4-956a48d16476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:21.401423 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1169/2329] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:21.413316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15ffff-6768-4042-8406-f81a32ff6db5 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:21.415153 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fa15ffff-6768-4042-8406-f81a32ff6db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:21.478888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15ffff-6768-4042-8406-f81a32ff6db5 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:21.482605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f818d-7bbc-45ce-ac8c-40655b91729b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:21.483264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f818d-7bbc-45ce-ac8c-40655b91729b service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:21.483543 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f818d-7bbc-45ce-ac8c-40655b91729b service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:21.483819 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f818d-7bbc-45ce-ac8c-40655b91729b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:21.548185 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f818d-7bbc-45ce-ac8c-40655b91729b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:21.549791 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f818d-7bbc-45ce-ac8c-40655b91729b service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:21.559199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1161/2330] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:17:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:23.734287 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3a92e-bd2d-49e9-842c-51dbe771d224 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:23.734864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3a92e-bd2d-49e9-842c-51dbe771d224 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:23.734864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3a92e-bd2d-49e9-842c-51dbe771d224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:23.755178 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6de3a92e-bd2d-49e9-842c-51dbe771d224 None None] MFA Rules not processed for user `ca1d32ee23bc42b38ea9a043f7a850f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:23.756767 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6de3a92e-bd2d-49e9-842c-51dbe771d224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:23.820585 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1170/2331] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:30.141410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea84649-9534-4508-9d91-49bc0f816936 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:30.141959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea84649-9534-4508-9d91-49bc0f816936 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:30.142059 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea84649-9534-4508-9d91-49bc0f816936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:30.162612 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-bea84649-9534-4508-9d91-49bc0f816936 None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:30.163883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bea84649-9534-4508-9d91-49bc0f816936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:30.199716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:30.236188 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1162/2332] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:17:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:30.289019 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3a92e-bd2d-49e9-842c-51dbe771d224 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:30.290495 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6de3a92e-bd2d-49e9-842c-51dbe771d224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:30.297978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:30.374837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3a92e-bd2d-49e9-842c-51dbe771d224 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:30.380481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be9916-2d34-4969-bcf2-c16044376457 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:30.381126 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be9916-2d34-4969-bcf2-c16044376457 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:30.381422 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be9916-2d34-4969-bcf2-c16044376457 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:30.381692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be9916-2d34-4969-bcf2-c16044376457 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:30.390978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be9916-2d34-4969-bcf2-c16044376457 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:30.392616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be9916-2d34-4969-bcf2-c16044376457 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:30.402045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1171/2333] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:17:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:17:31.778775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea84649-9534-4508-9d91-49bc0f816936 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:31.780664 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bea84649-9534-4508-9d91-49bc0f816936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:31.799039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea84649-9534-4508-9d91-49bc0f816936 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:31.804170 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aefaef-322f-4961-9922-0fe587c20306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:31.805244 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aefaef-322f-4961-9922-0fe587c20306 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:31.805700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aefaef-322f-4961-9922-0fe587c20306 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:31.806334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aefaef-322f-4961-9922-0fe587c20306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:31.936437 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aefaef-322f-4961-9922-0fe587c20306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:31.938428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aefaef-322f-4961-9922-0fe587c20306 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:31.955763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1163/2334] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:17:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:32.896098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be9916-2d34-4969-bcf2-c16044376457 service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:32.918514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be9916-2d34-4969-bcf2-c16044376457 service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:32.923739 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b145e8-10ca-4dbc-b0de-cb8d5f24d2a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:32.924688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b145e8-10ca-4dbc-b0de-cb8d5f24d2a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:32.925228 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b145e8-10ca-4dbc-b0de-cb8d5f24d2a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:32.925672 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b145e8-10ca-4dbc-b0de-cb8d5f24d2a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:32.937853 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b145e8-10ca-4dbc-b0de-cb8d5f24d2a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:32.940160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b145e8-10ca-4dbc-b0de-cb8d5f24d2a2 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:32.955493 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1172/2335] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:17:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:40.503734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab18dbc-84d6-4734-9d03-9ccfbc7773e4 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:40.504348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab18dbc-84d6-4734-9d03-9ccfbc7773e4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:40.504348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab18dbc-84d6-4734-9d03-9ccfbc7773e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:40.526801 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4ab18dbc-84d6-4734-9d03-9ccfbc7773e4 None None] MFA Rules not processed for user `c6795a827b8d411aaa165db7b38435ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:40.528111 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4ab18dbc-84d6-4734-9d03-9ccfbc7773e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:40.605781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1164/2336] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:42.948215 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=36 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:50.592305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=63 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:52.818302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b145e8-10ca-4dbc-b0de-cb8d5f24d2a2 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:52.820198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-83b145e8-10ca-4dbc-b0de-cb8d5f24d2a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:52.905874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b145e8-10ca-4dbc-b0de-cb8d5f24d2a2 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:52.913414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6287f-eb66-476c-98f6-52f14db8d452 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:52.914615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6287f-eb66-476c-98f6-52f14db8d452 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:52.915047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6287f-eb66-476c-98f6-52f14db8d452 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:52.915667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6287f-eb66-476c-98f6-52f14db8d452 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:52.917923 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8cd6287f-eb66-476c-98f6-52f14db8d452 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:52.941718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6287f-eb66-476c-98f6-52f14db8d452 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:52.944700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6287f-eb66-476c-98f6-52f14db8d452 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:52.958462 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1173/2337] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:17:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:53.324495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab18dbc-84d6-4734-9d03-9ccfbc7773e4 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:53.442370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab18dbc-84d6-4734-9d03-9ccfbc7773e4 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:53.447629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb613608-2fed-40ec-84f2-3f9603bf60ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:53.448513 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb613608-2fed-40ec-84f2-3f9603bf60ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:53.449008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb613608-2fed-40ec-84f2-3f9603bf60ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:53.449403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb613608-2fed-40ec-84f2-3f9603bf60ce admin admin] PATH_INFO: `/v3/users/4800a61ea4244630b92dc61d5d80d89c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:53.451306 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb613608-2fed-40ec-84f2-3f9603bf60ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=4800a61ea4244630b92dc61d5d80d89c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:53.454383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb613608-2fed-40ec-84f2-3f9603bf60ce admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:53.521322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-eb613608-2fed-40ec-84f2-3f9603bf60ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4800a61ea4244630b92dc61d5d80d89c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:53.537595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-eb613608-2fed-40ec-84f2-3f9603bf60ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4800a61ea4244630b92dc61d5d80d89c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:53.539056 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-eb613608-2fed-40ec-84f2-3f9603bf60ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4800a61ea4244630b92dc61d5d80d89c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:53.540287 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-eb613608-2fed-40ec-84f2-3f9603bf60ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4800a61ea4244630b92dc61d5d80d89c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:53.562343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-eb613608-2fed-40ec-84f2-3f9603bf60ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4800a61ea4244630b92dc61d5d80d89c'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:53.577266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1165/2338] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:17:53 2022] DELETE /identity/v3/users/4800a61ea4244630b92dc61d5d80d89c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:53.582548 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fa07b-7646-4bfb-9db5-fa1392fb455f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:53.582871 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fa07b-7646-4bfb-9db5-fa1392fb455f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:53.583153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fa07b-7646-4bfb-9db5-fa1392fb455f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:53.608906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c75fa07b-7646-4bfb-9db5-fa1392fb455f None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:53.611067 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c75fa07b-7646-4bfb-9db5-fa1392fb455f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:53.708271 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1174/2339] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:53.719295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb613608-2fed-40ec-84f2-3f9603bf60ce admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:53.721401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-eb613608-2fed-40ec-84f2-3f9603bf60ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:53.869274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb613608-2fed-40ec-84f2-3f9603bf60ce admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:53.879971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072a0e0-4cf9-4a42-b143-69fd71b3c7ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:53.881687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072a0e0-4cf9-4a42-b143-69fd71b3c7ca service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:53.882279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072a0e0-4cf9-4a42-b143-69fd71b3c7ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:53.882901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a072a0e0-4cf9-4a42-b143-69fd71b3c7ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:53.899733 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072a0e0-4cf9-4a42-b143-69fd71b3c7ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:53.901210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a072a0e0-4cf9-4a42-b143-69fd71b3c7ca service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:53.919816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1166/2340] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:17:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:17:54.424737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fa07b-7646-4bfb-9db5-fa1392fb455f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:54.426285 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c75fa07b-7646-4bfb-9db5-fa1392fb455f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:54.453804 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fa07b-7646-4bfb-9db5-fa1392fb455f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:54.519127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:54.521101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:54.522147 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:54.523381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] PATH_INFO: `/v3/projects/3d62512a132041a2be588d949ced06d0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:54.526269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d62512a132041a2be588d949ced06d0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:54.532733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:54.649393 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d62512a132041a2be588d949ced06d0', 'request_id': 'req-184fd8cf-177c-4a79-aa2a-f05a67640f85'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:54.660578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d62512a132041a2be588d949ced06d0', 'request_id': 'req-184fd8cf-177c-4a79-aa2a-f05a67640f85'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:54.677291 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d62512a132041a2be588d949ced06d0', 'request_id': 'req-184fd8cf-177c-4a79-aa2a-f05a67640f85'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:54.695509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] The token cache is being invalidate because project 3d62512a132041a2be588d949ced06d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:17:54.696054 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:54.699420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:54.702698 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1175/2341] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:17:54 2022] DELETE /identity/v3/projects/3d62512a132041a2be588d949ced06d0 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:54.706574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072a0e0-4cf9-4a42-b143-69fd71b3c7ca service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:54.717130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a072a0e0-4cf9-4a42-b143-69fd71b3c7ca service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:54.745569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:54.802616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3020b1b-e461-4c72-8114-f510edf3c377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:54.803244 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3020b1b-e461-4c72-8114-f510edf3c377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:54.803700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3020b1b-e461-4c72-8114-f510edf3c377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:54.804147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3020b1b-e461-4c72-8114-f510edf3c377 admin admin] PATH_INFO: `/v3/users/3f51428c41c94556855f850d047437bf` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:54.806369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3020b1b-e461-4c72-8114-f510edf3c377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f51428c41c94556855f850d047437bf)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:54.808536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3020b1b-e461-4c72-8114-f510edf3c377 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:54.886819 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b3020b1b-e461-4c72-8114-f510edf3c377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f51428c41c94556855f850d047437bf'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:54.912533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b3020b1b-e461-4c72-8114-f510edf3c377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f51428c41c94556855f850d047437bf'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:54.914529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b3020b1b-e461-4c72-8114-f510edf3c377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f51428c41c94556855f850d047437bf'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:54.915268 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b3020b1b-e461-4c72-8114-f510edf3c377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f51428c41c94556855f850d047437bf'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:54.942835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-b3020b1b-e461-4c72-8114-f510edf3c377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f51428c41c94556855f850d047437bf'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:54.956179 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1167/2342] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:17:54 2022] DELETE /identity/v3/users/3f51428c41c94556855f850d047437bf => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:55.284883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:55.286348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:55.309025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fd8cf-177c-4a79-aa2a-f05a67640f85 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:55.339622 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:55.431405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:55.432857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:55.433143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:55.433429 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] PATH_INFO: `/v3/projects/81279e65218f440da6d6810a58c4c5dd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:55.434731 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81279e65218f440da6d6810a58c4c5dd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:55.436643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:55.501365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81279e65218f440da6d6810a58c4c5dd', 'request_id': 'req-fde0ad74-559f-4a19-a3b1-bf4057266ff8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:55.514222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81279e65218f440da6d6810a58c4c5dd', 'request_id': 'req-fde0ad74-559f-4a19-a3b1-bf4057266ff8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:55.540249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81279e65218f440da6d6810a58c4c5dd', 'request_id': 'req-fde0ad74-559f-4a19-a3b1-bf4057266ff8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:55.568331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] The token cache is being invalidate because project 81279e65218f440da6d6810a58c4c5dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:17:55.568623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:55.569588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:55.571840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1176/2343] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:17:55 2022] DELETE /identity/v3/projects/81279e65218f440da6d6810a58c4c5dd => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:55.576715 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3020b1b-e461-4c72-8114-f510edf3c377 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:55.578640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b3020b1b-e461-4c72-8114-f510edf3c377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:55.604639 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3020b1b-e461-4c72-8114-f510edf3c377 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:55.733405 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905361e-54fe-4ba8-a838-9dfe6f19a3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:55.734559 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905361e-54fe-4ba8-a838-9dfe6f19a3de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:55.735197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905361e-54fe-4ba8-a838-9dfe6f19a3de admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:55.735778 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905361e-54fe-4ba8-a838-9dfe6f19a3de admin admin] PATH_INFO: `/v3/users/15daec108538449184a1ac5fff6d8909` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:55.738125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905361e-54fe-4ba8-a838-9dfe6f19a3de admin admin] RBAC: Authorizing `identity:delete_user(user_id=15daec108538449184a1ac5fff6d8909)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:55.740856 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905361e-54fe-4ba8-a838-9dfe6f19a3de admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:55.834373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:55.840256 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2905361e-54fe-4ba8-a838-9dfe6f19a3de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15daec108538449184a1ac5fff6d8909'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:55.878187 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2905361e-54fe-4ba8-a838-9dfe6f19a3de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15daec108538449184a1ac5fff6d8909'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:55.879170 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2905361e-54fe-4ba8-a838-9dfe6f19a3de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15daec108538449184a1ac5fff6d8909'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:55.880652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2905361e-54fe-4ba8-a838-9dfe6f19a3de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15daec108538449184a1ac5fff6d8909'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:55.922624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2905361e-54fe-4ba8-a838-9dfe6f19a3de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15daec108538449184a1ac5fff6d8909'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:55.946431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1168/2344] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:17:55 2022] DELETE /identity/v3/users/15daec108538449184a1ac5fff6d8909 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:56.453579 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:56.456512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:56.487771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0ad74-559f-4a19-a3b1-bf4057266ff8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:56.651913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4639b4a-8cda-44f6-8204-0bc370192d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:56.653227 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4639b4a-8cda-44f6-8204-0bc370192d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:56.653673 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4639b4a-8cda-44f6-8204-0bc370192d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:56.654107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4639b4a-8cda-44f6-8204-0bc370192d13 admin admin] PATH_INFO: `/v3/projects/791399b2833247ea96bb569dc648d831` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:56.655662 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4639b4a-8cda-44f6-8204-0bc370192d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=791399b2833247ea96bb569dc648d831)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:56.659134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4639b4a-8cda-44f6-8204-0bc370192d13 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:56.733684 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:56.738496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b4639b4a-8cda-44f6-8204-0bc370192d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '791399b2833247ea96bb569dc648d831', 'request_id': 'req-b4639b4a-8cda-44f6-8204-0bc370192d13'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:56.749257 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b4639b4a-8cda-44f6-8204-0bc370192d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '791399b2833247ea96bb569dc648d831', 'request_id': 'req-b4639b4a-8cda-44f6-8204-0bc370192d13'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:56.757097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b4639b4a-8cda-44f6-8204-0bc370192d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '791399b2833247ea96bb569dc648d831', 'request_id': 'req-b4639b4a-8cda-44f6-8204-0bc370192d13'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:56.789280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b4639b4a-8cda-44f6-8204-0bc370192d13 admin admin] The token cache is being invalidate because project 791399b2833247ea96bb569dc648d831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:17:56.789971 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b4639b4a-8cda-44f6-8204-0bc370192d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:56.792110 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b4639b4a-8cda-44f6-8204-0bc370192d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:17:56.800250 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f68a65-e35a-4ab9-bf21-32d4dd5e34d0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:56.801302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f68a65-e35a-4ab9-bf21-32d4dd5e34d0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:56.801748 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f68a65-e35a-4ab9-bf21-32d4dd5e34d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:56.806122 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1177/2345] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:17:56 2022] DELETE /identity/v3/projects/791399b2833247ea96bb569dc648d831 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:56.840882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c5f68a65-e35a-4ab9-bf21-32d4dd5e34d0 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:56.843166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c5f68a65-e35a-4ab9-bf21-32d4dd5e34d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:56.933921 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1169/2346] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:56.943922 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a07ad5-aa66-4b5c-912d-79ac6e997806 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:56.944306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a07ad5-aa66-4b5c-912d-79ac6e997806 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:56.944690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a07ad5-aa66-4b5c-912d-79ac6e997806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:56.970806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d7a07ad5-aa66-4b5c-912d-79ac6e997806 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:56.972347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d7a07ad5-aa66-4b5c-912d-79ac6e997806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:56.988334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1178/2347] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:56.997983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f68a65-e35a-4ab9-bf21-32d4dd5e34d0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:57.011698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f68a65-e35a-4ab9-bf21-32d4dd5e34d0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:57.027184 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab2ab8-a0be-4a08-aa85-de1a9f25ab9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:57.028563 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab2ab8-a0be-4a08-aa85-de1a9f25ab9b admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:57.028908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab2ab8-a0be-4a08-aa85-de1a9f25ab9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:57.029200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfab2ab8-a0be-4a08-aa85-de1a9f25ab9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:57.030047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab2ab8-a0be-4a08-aa85-de1a9f25ab9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:17:57.030546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab2ab8-a0be-4a08-aa85-de1a9f25ab9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:57.032710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfab2ab8-a0be-4a08-aa85-de1a9f25ab9b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:57.070938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1170/2348] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:17:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:17:57.075086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a07ad5-aa66-4b5c-912d-79ac6e997806 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:57.087705 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a07ad5-aa66-4b5c-912d-79ac6e997806 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:57.093214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de784836-c9a0-44aa-983d-27ce50e89adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:57.094095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de784836-c9a0-44aa-983d-27ce50e89adf admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:57.094381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de784836-c9a0-44aa-983d-27ce50e89adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:57.095267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de784836-c9a0-44aa-983d-27ce50e89adf admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:57.096873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de784836-c9a0-44aa-983d-27ce50e89adf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:57.099130 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de784836-c9a0-44aa-983d-27ce50e89adf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:57.140368 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1179/2349] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:17:57 2022] POST /identity/v3/projects => generated 374 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:17:57.141891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab2ab8-a0be-4a08-aa85-de1a9f25ab9b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:57.158170 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfab2ab8-a0be-4a08-aa85-de1a9f25ab9b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:57.276764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3537de-7bdf-4fff-9586-1f9adbd303c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:57.277411 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3537de-7bdf-4fff-9586-1f9adbd303c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:57.277701 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3537de-7bdf-4fff-9586-1f9adbd303c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:57.277982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3537de-7bdf-4fff-9586-1f9adbd303c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:57.278688 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3537de-7bdf-4fff-9586-1f9adbd303c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:57.288985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3537de-7bdf-4fff-9586-1f9adbd303c9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:57.323497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1171/2350] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:17:57 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:17:57.327517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de784836-c9a0-44aa-983d-27ce50e89adf admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:57.342470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de784836-c9a0-44aa-983d-27ce50e89adf admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:57.350405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b82a1-3104-433e-aaeb-63aa2d09d221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:57.351272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b82a1-3104-433e-aaeb-63aa2d09d221 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:57.351559 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b82a1-3104-433e-aaeb-63aa2d09d221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:57.351842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b82a1-3104-433e-aaeb-63aa2d09d221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:57.352685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b82a1-3104-433e-aaeb-63aa2d09d221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:57.354351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b82a1-3104-433e-aaeb-63aa2d09d221 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:57.366486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1180/2351] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:57 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:17:57.370339 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3537de-7bdf-4fff-9586-1f9adbd303c9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:57.384617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3537de-7bdf-4fff-9586-1f9adbd303c9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:57.390209 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f434e-1ec7-4a86-9e20-03c81ab43a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:57.390892 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f434e-1ec7-4a86-9e20-03c81ab43a9e admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:57.391176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f434e-1ec7-4a86-9e20-03c81ab43a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:57.391457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f434e-1ec7-4a86-9e20-03c81ab43a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:57.392164 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f434e-1ec7-4a86-9e20-03c81ab43a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:57.394398 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3f434e-1ec7-4a86-9e20-03c81ab43a9e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:57.410112 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1172/2352] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:17:57 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:17:57.414545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b82a1-3104-433e-aaeb-63aa2d09d221 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:57.429475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b82a1-3104-433e-aaeb-63aa2d09d221 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:57.435885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5ecf3-5d7b-42db-a09b-2b748a213f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:57.436863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5ecf3-5d7b-42db-a09b-2b748a213f22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:57.437353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5ecf3-5d7b-42db-a09b-2b748a213f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:57.438033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5ecf3-5d7b-42db-a09b-2b748a213f22 admin admin] PATH_INFO: `/v3/projects/d1bee8437e854294b8e1186391e52aa6/users/5d2c6ceff9d74f4e8c8194f45e52dc0b/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:57.462027 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5ecf3-5d7b-42db-a09b-2b748a213f22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1bee8437e854294b8e1186391e52aa6, user_id=5d2c6ceff9d74f4e8c8194f45e52dc0b, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:57.464825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5ecf3-5d7b-42db-a09b-2b748a213f22 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:57.482757 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1181/2353] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:17:57 2022] PUT /identity/v3/projects/d1bee8437e854294b8e1186391e52aa6/users/5d2c6ceff9d74f4e8c8194f45e52dc0b/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:17:57.493100 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96077ca6-c6c2-4a19-9b4f-d76d25a71777 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:57.493481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96077ca6-c6c2-4a19-9b4f-d76d25a71777 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:57.493772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96077ca6-c6c2-4a19-9b4f-d76d25a71777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:57.544290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-96077ca6-c6c2-4a19-9b4f-d76d25a71777 None None] MFA Rules not processed for user `5d2c6ceff9d74f4e8c8194f45e52dc0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:17:57.545608 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-96077ca6-c6c2-4a19-9b4f-d76d25a71777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:57.567764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:17:57.647135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1173/2354] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:17:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:17:57.659120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5ecf3-5d7b-42db-a09b-2b748a213f22 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:17:57.660906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-60d5ecf3-5d7b-42db-a09b-2b748a213f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:17:57.739926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5ecf3-5d7b-42db-a09b-2b748a213f22 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:17:57.744773 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd73a61-c437-4380-98f9-8a8d7e0064a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:17:57.745471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd73a61-c437-4380-98f9-8a8d7e0064a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:17:57.745766 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd73a61-c437-4380-98f9-8a8d7e0064a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:17:57.746057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd73a61-c437-4380-98f9-8a8d7e0064a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:17:57.756801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd73a61-c437-4380-98f9-8a8d7e0064a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:17:57.758347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd73a61-c437-4380-98f9-8a8d7e0064a3 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:17:57.769638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1182/2355] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:17:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:00.341067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce1f03-ac0f-4149-af47-cc17c76d1aba None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:00.341913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce1f03-ac0f-4149-af47-cc17c76d1aba None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:00.341913 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce1f03-ac0f-4149-af47-cc17c76d1aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:00.364760 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d0ce1f03-ac0f-4149-af47-cc17c76d1aba None None] MFA Rules not processed for user `5d2c6ceff9d74f4e8c8194f45e52dc0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:00.366290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d0ce1f03-ac0f-4149-af47-cc17c76d1aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:00.389891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1174/2356] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:18:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:18:01.622593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036cdbf-26a7-487b-851b-fe1c741c1524 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:01.623597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036cdbf-26a7-487b-851b-fe1c741c1524 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:01.624064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036cdbf-26a7-487b-851b-fe1c741c1524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:01.667359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e036cdbf-26a7-487b-851b-fe1c741c1524 None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:01.674963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e036cdbf-26a7-487b-851b-fe1c741c1524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:01.794552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1183/2357] 10.222.0.100 () {58 vars in 972 bytes} [Mon Aug 1 15:18:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:01.812241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce1f03-ac0f-4149-af47-cc17c76d1aba None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:01.815290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d0ce1f03-ac0f-4149-af47-cc17c76d1aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:01.930165 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce1f03-ac0f-4149-af47-cc17c76d1aba None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:01.935401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000190f-36f3-4783-af8a-4d2150e9e9e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:01.936080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000190f-36f3-4783-af8a-4d2150e9e9e3 service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:01.936364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000190f-36f3-4783-af8a-4d2150e9e9e3 service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:01.936644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000190f-36f3-4783-af8a-4d2150e9e9e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:01.947276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a000190f-36f3-4783-af8a-4d2150e9e9e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:01.948964 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a000190f-36f3-4783-af8a-4d2150e9e9e3 service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:01.960176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1175/2358] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:18:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:18:04.736472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036cdbf-26a7-487b-851b-fe1c741c1524 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:04.738551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e036cdbf-26a7-487b-851b-fe1c741c1524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:04.780610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:04.845135 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036cdbf-26a7-487b-851b-fe1c741c1524 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:04.850373 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5948c2-b953-4e71-9a4b-f25542366d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:04.851417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5948c2-b953-4e71-9a4b-f25542366d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:04.851694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5948c2-b953-4e71-9a4b-f25542366d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:04.851974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5948c2-b953-4e71-9a4b-f25542366d58 admin admin] PATH_INFO: `/v3/users/55bb7887d98f458688cffbccf4a64b53` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:04.853395 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5948c2-b953-4e71-9a4b-f25542366d58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55bb7887d98f458688cffbccf4a64b53)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:04.855267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5948c2-b953-4e71-9a4b-f25542366d58 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:04.922517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b5948c2-b953-4e71-9a4b-f25542366d58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55bb7887d98f458688cffbccf4a64b53'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:04.923275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b5948c2-b953-4e71-9a4b-f25542366d58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55bb7887d98f458688cffbccf4a64b53'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:04.933354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b5948c2-b953-4e71-9a4b-f25542366d58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55bb7887d98f458688cffbccf4a64b53'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:04.934595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b5948c2-b953-4e71-9a4b-f25542366d58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55bb7887d98f458688cffbccf4a64b53'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:04.949654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b5948c2-b953-4e71-9a4b-f25542366d58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55bb7887d98f458688cffbccf4a64b53'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:04.966087 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1184/2359] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:18:04 2022] DELETE /identity/v3/users/55bb7887d98f458688cffbccf4a64b53 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:05.182979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000190f-36f3-4783-af8a-4d2150e9e9e3 service swift] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:05.201430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000190f-36f3-4783-af8a-4d2150e9e9e3 service swift] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:05.301441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee22b5c-9f37-4ed9-9785-c24950711c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:05.302420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee22b5c-9f37-4ed9-9785-c24950711c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:05.302771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee22b5c-9f37-4ed9-9785-c24950711c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:05.303143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee22b5c-9f37-4ed9-9785-c24950711c91 admin admin] PATH_INFO: `/v3/projects/cc38647adf7e41beb5c47c01149972e0` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:05.304949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee22b5c-9f37-4ed9-9785-c24950711c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc38647adf7e41beb5c47c01149972e0)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:05.307630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee22b5c-9f37-4ed9-9785-c24950711c91 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:05.370286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:05.386064 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6ee22b5c-9f37-4ed9-9785-c24950711c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc38647adf7e41beb5c47c01149972e0', 'request_id': 'req-6ee22b5c-9f37-4ed9-9785-c24950711c91'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:05.393638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6ee22b5c-9f37-4ed9-9785-c24950711c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc38647adf7e41beb5c47c01149972e0', 'request_id': 'req-6ee22b5c-9f37-4ed9-9785-c24950711c91'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:05.403577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6ee22b5c-9f37-4ed9-9785-c24950711c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc38647adf7e41beb5c47c01149972e0', 'request_id': 'req-6ee22b5c-9f37-4ed9-9785-c24950711c91'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:05.418738 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6ee22b5c-9f37-4ed9-9785-c24950711c91 admin admin] The token cache is being invalidate because project cc38647adf7e41beb5c47c01149972e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:18:05.419176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6ee22b5c-9f37-4ed9-9785-c24950711c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:05.419989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6ee22b5c-9f37-4ed9-9785-c24950711c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:05.422514 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1176/2360] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:18:05 2022] DELETE /identity/v3/projects/cc38647adf7e41beb5c47c01149972e0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:05.428918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6f0b0-1a15-430e-9dd4-0e303746516d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:05.428918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6f0b0-1a15-430e-9dd4-0e303746516d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:05.428918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6f0b0-1a15-430e-9dd4-0e303746516d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:05.462459 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f4b6f0b0-1a15-430e-9dd4-0e303746516d None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:05.463777 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f4b6f0b0-1a15-430e-9dd4-0e303746516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:05.563013 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1185/2361] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:05.565866 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee72fcd-8c61-4d04-aa68-4d5e0e48786a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:05.566594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee72fcd-8c61-4d04-aa68-4d5e0e48786a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:05.567242 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee72fcd-8c61-4d04-aa68-4d5e0e48786a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:05.591403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-fee72fcd-8c61-4d04-aa68-4d5e0e48786a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:05.592836 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fee72fcd-8c61-4d04-aa68-4d5e0e48786a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:05.606307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1177/2362] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:05.607356 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6f0b0-1a15-430e-9dd4-0e303746516d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:05.618498 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6f0b0-1a15-430e-9dd4-0e303746516d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:05.630701 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43473d8-473d-487b-b8cb-a25c5d5030c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:05.631442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43473d8-473d-487b-b8cb-a25c5d5030c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:05.631692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43473d8-473d-487b-b8cb-a25c5d5030c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:05.631958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43473d8-473d-487b-b8cb-a25c5d5030c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:05.632832 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43473d8-473d-487b-b8cb-a25c5d5030c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:18:05.633118 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43473d8-473d-487b-b8cb-a25c5d5030c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:05.635023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43473d8-473d-487b-b8cb-a25c5d5030c5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:05.660991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1186/2363] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:18:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:18:05.663014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee72fcd-8c61-4d04-aa68-4d5e0e48786a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:05.676348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee72fcd-8c61-4d04-aa68-4d5e0e48786a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:05.683073 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef823d-2724-41a3-bb72-ef601546a6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:05.683901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef823d-2724-41a3-bb72-ef601546a6c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:05.684242 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef823d-2724-41a3-bb72-ef601546a6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:05.685446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef823d-2724-41a3-bb72-ef601546a6c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:05.685446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef823d-2724-41a3-bb72-ef601546a6c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:05.687303 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef823d-2724-41a3-bb72-ef601546a6c8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:05.743294 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1178/2364] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:18:05 2022] POST /identity/v3/projects => generated 370 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:18:05.748510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43473d8-473d-487b-b8cb-a25c5d5030c5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:05.769574 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43473d8-473d-487b-b8cb-a25c5d5030c5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:05.796588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:05.866747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a798e-bf18-4b11-afb9-91ffd8b7213b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:05.867496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a798e-bf18-4b11-afb9-91ffd8b7213b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:05.867893 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a798e-bf18-4b11-afb9-91ffd8b7213b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:05.868652 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a798e-bf18-4b11-afb9-91ffd8b7213b admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:05.869417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a798e-bf18-4b11-afb9-91ffd8b7213b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:05.871584 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a798e-bf18-4b11-afb9-91ffd8b7213b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:05.900832 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1187/2365] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:18:05 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:18:05.905095 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef823d-2724-41a3-bb72-ef601546a6c8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:05.916423 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef823d-2724-41a3-bb72-ef601546a6c8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:05.921232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def5e70-6520-4684-ac06-de7342396e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:05.922006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def5e70-6520-4684-ac06-de7342396e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:05.922299 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def5e70-6520-4684-ac06-de7342396e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:05.922577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def5e70-6520-4684-ac06-de7342396e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:05.923516 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def5e70-6520-4684-ac06-de7342396e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:05.932393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def5e70-6520-4684-ac06-de7342396e02 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:05.945703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1179/2366] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:18:05 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:18:05.950133 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a798e-bf18-4b11-afb9-91ffd8b7213b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:05.964436 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a798e-bf18-4b11-afb9-91ffd8b7213b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:05.970042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3735e-6161-43e9-9a03-c45c23e2db2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:05.971334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3735e-6161-43e9-9a03-c45c23e2db2c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:05.971624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3735e-6161-43e9-9a03-c45c23e2db2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:05.971905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3735e-6161-43e9-9a03-c45c23e2db2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:05.972620 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3735e-6161-43e9-9a03-c45c23e2db2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:05.974772 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3735e-6161-43e9-9a03-c45c23e2db2c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:05.990029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1188/2367] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:18:05 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:18:05.996382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def5e70-6520-4684-ac06-de7342396e02 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:06.008969 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def5e70-6520-4684-ac06-de7342396e02 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:06.014134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494bc60-ed7a-452e-a38d-eb4d65834683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:06.014856 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494bc60-ed7a-452e-a38d-eb4d65834683 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:06.015143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494bc60-ed7a-452e-a38d-eb4d65834683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:06.015428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494bc60-ed7a-452e-a38d-eb4d65834683 admin admin] PATH_INFO: `/v3/projects/4bc4f659ce964d4bb1c5b5ab9007c93a/users/7c8bf6c0ae2c4fa5bf36ff3bfa77901d/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:06.033974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494bc60-ed7a-452e-a38d-eb4d65834683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bc4f659ce964d4bb1c5b5ab9007c93a, user_id=7c8bf6c0ae2c4fa5bf36ff3bfa77901d, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:06.036728 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494bc60-ed7a-452e-a38d-eb4d65834683 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:06.051306 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1180/2368] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:18:05 2022] PUT /identity/v3/projects/4bc4f659ce964d4bb1c5b5ab9007c93a/users/7c8bf6c0ae2c4fa5bf36ff3bfa77901d/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:06.061957 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd957c7-d9b9-4b20-96ea-e5f066df7ac8 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:06.062308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd957c7-d9b9-4b20-96ea-e5f066df7ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:06.062934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd957c7-d9b9-4b20-96ea-e5f066df7ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:06.100178 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1dd957c7-d9b9-4b20-96ea-e5f066df7ac8 None None] MFA Rules not processed for user `7c8bf6c0ae2c4fa5bf36ff3bfa77901d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:06.101421 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1dd957c7-d9b9-4b20-96ea-e5f066df7ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:06.174967 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1189/2369] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:18:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:06.187186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494bc60-ed7a-452e-a38d-eb4d65834683 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:06.188923 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b494bc60-ed7a-452e-a38d-eb4d65834683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:06.255201 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494bc60-ed7a-452e-a38d-eb4d65834683 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:06.259189 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebf681-2fdb-4fbc-b5b1-b89d65dd35d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e75edfc5bb404d63b27a355d5ab8af01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:06.259828 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebf681-2fdb-4fbc-b5b1-b89d65dd35d0 service glance] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:06.260113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebf681-2fdb-4fbc-b5b1-b89d65dd35d0 service glance] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:06.260391 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebf681-2fdb-4fbc-b5b1-b89d65dd35d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:06.272529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebf681-2fdb-4fbc-b5b1-b89d65dd35d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:06.274828 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ebf681-2fdb-4fbc-b5b1-b89d65dd35d0 service glance] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:06.286206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1181/2370] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:18:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:18:16.172221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=55,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:16.281312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=62,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:20.583808 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56befa51-1d28-405c-a78d-01164a0f4b84 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:20.584387 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56befa51-1d28-405c-a78d-01164a0f4b84 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:20.584562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56befa51-1d28-405c-a78d-01164a0f4b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:20.605783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-56befa51-1d28-405c-a78d-01164a0f4b84 None None] MFA Rules not processed for user `b507df75a6564522bf6746587d081a03`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:20.607083 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-56befa51-1d28-405c-a78d-01164a0f4b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:20.670559 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1190/2371] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:18:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:27.945768 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebf681-2fdb-4fbc-b5b1-b89d65dd35d0 service glance] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:27.947266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d6ebf681-2fdb-4fbc-b5b1-b89d65dd35d0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:28.017824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebf681-2fdb-4fbc-b5b1-b89d65dd35d0 service glance] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:28.021620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5224c76-49d8-49fd-89e5-ad1eb23dda56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:28.022253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5224c76-49d8-49fd-89e5-ad1eb23dda56 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:28.022546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5224c76-49d8-49fd-89e5-ad1eb23dda56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:28.022855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5224c76-49d8-49fd-89e5-ad1eb23dda56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:28.024669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a5224c76-49d8-49fd-89e5-ad1eb23dda56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:28.090971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5224c76-49d8-49fd-89e5-ad1eb23dda56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:28.092428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5224c76-49d8-49fd-89e5-ad1eb23dda56 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:28.103177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1182/2372] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:18:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:18:30.663616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=25 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:38.098366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=50 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:48.737710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b881add-ab37-422e-8fd6-64e0c39cb7ac None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:48.738256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b881add-ab37-422e-8fd6-64e0c39cb7ac None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:48.738358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b881add-ab37-422e-8fd6-64e0c39cb7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:48.759033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4b881add-ab37-422e-8fd6-64e0c39cb7ac None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:48.760926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4b881add-ab37-422e-8fd6-64e0c39cb7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:48.820829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1191/2373] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:18:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:48.843101 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5224c76-49d8-49fd-89e5-ad1eb23dda56 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:48.845317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a5224c76-49d8-49fd-89e5-ad1eb23dda56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:48.915364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5224c76-49d8-49fd-89e5-ad1eb23dda56 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:48.920420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57ca6e-459a-4b98-a101-0ad0d0a4b1b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:48.921046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57ca6e-459a-4b98-a101-0ad0d0a4b1b0 service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:48.921332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57ca6e-459a-4b98-a101-0ad0d0a4b1b0 service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:48.921641 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57ca6e-459a-4b98-a101-0ad0d0a4b1b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:48.930702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57ca6e-459a-4b98-a101-0ad0d0a4b1b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:48.932227 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57ca6e-459a-4b98-a101-0ad0d0a4b1b0 service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:48.941096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1183/2374] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:18:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:18:49.473277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf95632-06c2-4c4d-9c4d-93b319f9feec None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:49.473575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf95632-06c2-4c4d-9c4d-93b319f9feec None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:49.473863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf95632-06c2-4c4d-9c4d-93b319f9feec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:49.495644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ddf95632-06c2-4c4d-9c4d-93b319f9feec None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:49.496872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ddf95632-06c2-4c4d-9c4d-93b319f9feec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:49.504614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1192/2375] 10.222.0.100 () {58 vars in 972 bytes} [Mon Aug 1 15:18:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:49.517593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57ca6e-459a-4b98-a101-0ad0d0a4b1b0 service swift] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:49.529121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57ca6e-459a-4b98-a101-0ad0d0a4b1b0 service swift] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:49.535480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a94c97-c29d-4e47-a2a9-71fec698f977 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:49.536104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a94c97-c29d-4e47-a2a9-71fec698f977 service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:49.536388 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a94c97-c29d-4e47-a2a9-71fec698f977 service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:49.536660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a94c97-c29d-4e47-a2a9-71fec698f977 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:49.551817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a94c97-c29d-4e47-a2a9-71fec698f977 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:49.553486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a94c97-c29d-4e47-a2a9-71fec698f977 service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:49.565532 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1184/2376] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:18:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:18:49.836491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df715529-b2fa-4204-b99e-39caef57f497 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:49.836996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df715529-b2fa-4204-b99e-39caef57f497 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:49.837079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df715529-b2fa-4204-b99e-39caef57f497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:49.857995 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-df715529-b2fa-4204-b99e-39caef57f497 None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:49.859808 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-df715529-b2fa-4204-b99e-39caef57f497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:49.867111 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1193/2377] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:18:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:49.879597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a94c97-c29d-4e47-a2a9-71fec698f977 service swift] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:49.891747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a94c97-c29d-4e47-a2a9-71fec698f977 service swift] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:49.897095 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e376f2-a02c-4d56-b38b-81978763c232 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:49.897716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e376f2-a02c-4d56-b38b-81978763c232 service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:49.898001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e376f2-a02c-4d56-b38b-81978763c232 service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:49.898275 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e376f2-a02c-4d56-b38b-81978763c232 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:49.909531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e376f2-a02c-4d56-b38b-81978763c232 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:49.911211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e376f2-a02c-4d56-b38b-81978763c232 service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:49.919903 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1185/2378] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:18:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:18:50.250808 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4edf3e-6ac3-436a-a5a3-bc5118275f8e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:50.251100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4edf3e-6ac3-436a-a5a3-bc5118275f8e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:50.251416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4edf3e-6ac3-436a-a5a3-bc5118275f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:50.270732 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7e4edf3e-6ac3-436a-a5a3-bc5118275f8e None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:50.271969 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7e4edf3e-6ac3-436a-a5a3-bc5118275f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:50.279279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1194/2379] 10.222.0.100 () {58 vars in 972 bytes} [Mon Aug 1 15:18:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:50.291447 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e376f2-a02c-4d56-b38b-81978763c232 service swift] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:50.302806 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e376f2-a02c-4d56-b38b-81978763c232 service swift] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:50.308104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66e4b9-ae97-4a96-9990-7b704ea39bcb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:50.308717 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66e4b9-ae97-4a96-9990-7b704ea39bcb service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:50.309153 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66e4b9-ae97-4a96-9990-7b704ea39bcb service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:50.309434 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66e4b9-ae97-4a96-9990-7b704ea39bcb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:50.320226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66e4b9-ae97-4a96-9990-7b704ea39bcb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:50.321815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66e4b9-ae97-4a96-9990-7b704ea39bcb service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:50.330980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1186/2380] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:18:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:18:52.513757 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4edf3e-6ac3-436a-a5a3-bc5118275f8e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:52.515887 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7e4edf3e-6ac3-436a-a5a3-bc5118275f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:52.586304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4edf3e-6ac3-436a-a5a3-bc5118275f8e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:52.591095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e49ad-fa9d-4a55-abc4-a33cfbe29ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:52.591750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e49ad-fa9d-4a55-abc4-a33cfbe29ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:52.592033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e49ad-fa9d-4a55-abc4-a33cfbe29ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:52.592309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e49ad-fa9d-4a55-abc4-a33cfbe29ebd admin admin] PATH_INFO: `/v3/users/5d2c6ceff9d74f4e8c8194f45e52dc0b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:52.594108 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e49ad-fa9d-4a55-abc4-a33cfbe29ebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d2c6ceff9d74f4e8c8194f45e52dc0b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:52.596114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e49ad-fa9d-4a55-abc4-a33cfbe29ebd admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:52.648197 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ed9e49ad-fa9d-4a55-abc4-a33cfbe29ebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d2c6ceff9d74f4e8c8194f45e52dc0b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:52.648863 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ed9e49ad-fa9d-4a55-abc4-a33cfbe29ebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d2c6ceff9d74f4e8c8194f45e52dc0b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:52.656950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ed9e49ad-fa9d-4a55-abc4-a33cfbe29ebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d2c6ceff9d74f4e8c8194f45e52dc0b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:52.657635 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ed9e49ad-fa9d-4a55-abc4-a33cfbe29ebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d2c6ceff9d74f4e8c8194f45e52dc0b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:52.671398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ed9e49ad-fa9d-4a55-abc4-a33cfbe29ebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d2c6ceff9d74f4e8c8194f45e52dc0b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:52.684853 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1195/2381] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:18:52 2022] DELETE /identity/v3/users/5d2c6ceff9d74f4e8c8194f45e52dc0b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:18:52.690999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e272f14-d50c-40bc-9b22-ab5f003fd646 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:52.691489 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e272f14-d50c-40bc-9b22-ab5f003fd646 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:52.691729 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e272f14-d50c-40bc-9b22-ab5f003fd646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:52.712540 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-3e272f14-d50c-40bc-9b22-ab5f003fd646 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:18:52.713770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3e272f14-d50c-40bc-9b22-ab5f003fd646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:52.783386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1187/2382] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:18:52.793349 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e49ad-fa9d-4a55-abc4-a33cfbe29ebd admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:52.794843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ed9e49ad-fa9d-4a55-abc4-a33cfbe29ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:52.801379 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:52.861921 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e49ad-fa9d-4a55-abc4-a33cfbe29ebd admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:52.866982 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e60e8-2a8d-4597-aef4-16dce8f98a5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:52.867940 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e60e8-2a8d-4597-aef4-16dce8f98a5b service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:52.868422 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e60e8-2a8d-4597-aef4-16dce8f98a5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:52.868818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e60e8-2a8d-4597-aef4-16dce8f98a5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:52.880780 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e60e8-2a8d-4597-aef4-16dce8f98a5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:52.882376 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e60e8-2a8d-4597-aef4-16dce8f98a5b service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:52.893524 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1196/2383] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:18:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:18:53.096845 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e272f14-d50c-40bc-9b22-ab5f003fd646 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:18:53.098443 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3e272f14-d50c-40bc-9b22-ab5f003fd646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:18:53.112475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e272f14-d50c-40bc-9b22-ab5f003fd646 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:18:53.117367 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9904c-a8a8-490d-9507-82df569d8e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:18:53.118029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9904c-a8a8-490d-9507-82df569d8e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:18:53.118311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9904c-a8a8-490d-9507-82df569d8e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:18:53.118586 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9904c-a8a8-490d-9507-82df569d8e46 admin admin] PATH_INFO: `/v3/projects/d1bee8437e854294b8e1186391e52aa6` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:18:53.119798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd9904c-a8a8-490d-9507-82df569d8e46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1bee8437e854294b8e1186391e52aa6)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:18:53.121720 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd9904c-a8a8-490d-9507-82df569d8e46 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:18:53.167999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:18:53.171790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2cd9904c-a8a8-490d-9507-82df569d8e46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1bee8437e854294b8e1186391e52aa6', 'request_id': 'req-2cd9904c-a8a8-490d-9507-82df569d8e46'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:53.179406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2cd9904c-a8a8-490d-9507-82df569d8e46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1bee8437e854294b8e1186391e52aa6', 'request_id': 'req-2cd9904c-a8a8-490d-9507-82df569d8e46'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:53.187868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2cd9904c-a8a8-490d-9507-82df569d8e46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1bee8437e854294b8e1186391e52aa6', 'request_id': 'req-2cd9904c-a8a8-490d-9507-82df569d8e46'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:53.198434 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2cd9904c-a8a8-490d-9507-82df569d8e46 admin admin] The token cache is being invalidate because project d1bee8437e854294b8e1186391e52aa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:18:53.198764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2cd9904c-a8a8-490d-9507-82df569d8e46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:53.199398 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-2cd9904c-a8a8-490d-9507-82df569d8e46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:18:53.201235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1188/2384] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:18:53 2022] DELETE /identity/v3/projects/d1bee8437e854294b8e1186391e52aa6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:02.888430 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=24 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:03.195481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:19.029735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e60e8-2a8d-4597-aef4-16dce8f98a5b service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:19.031328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e74e60e8-2a8d-4597-aef4-16dce8f98a5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:19.044213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e60e8-2a8d-4597-aef4-16dce8f98a5b service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:19.105106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b71ba1-4ca5-4ce7-a6ce-6ec17b52c8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:19.105759 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b71ba1-4ca5-4ce7-a6ce-6ec17b52c8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.106044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b71ba1-4ca5-4ce7-a6ce-6ec17b52c8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.106324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b71ba1-4ca5-4ce7-a6ce-6ec17b52c8be admin admin] PATH_INFO: `/v3/users/7c8bf6c0ae2c4fa5bf36ff3bfa77901d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.107529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b71ba1-4ca5-4ce7-a6ce-6ec17b52c8be admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c8bf6c0ae2c4fa5bf36ff3bfa77901d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:19.109473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b71ba1-4ca5-4ce7-a6ce-6ec17b52c8be admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:19.162143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b7b71ba1-4ca5-4ce7-a6ce-6ec17b52c8be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c8bf6c0ae2c4fa5bf36ff3bfa77901d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:19.162891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b7b71ba1-4ca5-4ce7-a6ce-6ec17b52c8be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c8bf6c0ae2c4fa5bf36ff3bfa77901d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:19.170641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b7b71ba1-4ca5-4ce7-a6ce-6ec17b52c8be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c8bf6c0ae2c4fa5bf36ff3bfa77901d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:19.171370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b7b71ba1-4ca5-4ce7-a6ce-6ec17b52c8be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c8bf6c0ae2c4fa5bf36ff3bfa77901d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:19.184369 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-b7b71ba1-4ca5-4ce7-a6ce-6ec17b52c8be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c8bf6c0ae2c4fa5bf36ff3bfa77901d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:19.200633 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1197/2385] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:19:19 2022] DELETE /identity/v3/users/7c8bf6c0ae2c4fa5bf36ff3bfa77901d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:19.206825 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dcfd5-d2fa-4f42-a8eb-baf93033e024 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.207371 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dcfd5-d2fa-4f42-a8eb-baf93033e024 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.207634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10dcfd5-d2fa-4f42-a8eb-baf93033e024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.228772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f10dcfd5-d2fa-4f42-a8eb-baf93033e024 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:19.230155 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f10dcfd5-d2fa-4f42-a8eb-baf93033e024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:19.290998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1189/2386] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:19.300835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b71ba1-4ca5-4ce7-a6ce-6ec17b52c8be admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:19.302254 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b7b71ba1-4ca5-4ce7-a6ce-6ec17b52c8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:19.361094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b71ba1-4ca5-4ce7-a6ce-6ec17b52c8be admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:19.364607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760ee27-d807-4b00-aad7-f1640d635d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:19.365223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760ee27-d807-4b00-aad7-f1640d635d29 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.365500 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760ee27-d807-4b00-aad7-f1640d635d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.365772 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760ee27-d807-4b00-aad7-f1640d635d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.375872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760ee27-d807-4b00-aad7-f1640d635d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:19.377406 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760ee27-d807-4b00-aad7-f1640d635d29 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:19.387118 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1198/2387] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:19.668174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dcfd5-d2fa-4f42-a8eb-baf93033e024 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:19.670818 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f10dcfd5-d2fa-4f42-a8eb-baf93033e024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:19.687843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10dcfd5-d2fa-4f42-a8eb-baf93033e024 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:19.694893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8c7ef-032b-4b35-8941-adbf2ab16f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:19.695505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8c7ef-032b-4b35-8941-adbf2ab16f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.695682 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8c7ef-032b-4b35-8941-adbf2ab16f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.696035 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8c7ef-032b-4b35-8941-adbf2ab16f92 admin admin] PATH_INFO: `/v3/projects/4bc4f659ce964d4bb1c5b5ab9007c93a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.697100 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8c7ef-032b-4b35-8941-adbf2ab16f92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bc4f659ce964d4bb1c5b5ab9007c93a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:19.699122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8c7ef-032b-4b35-8941-adbf2ab16f92 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:19.748898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-84b8c7ef-032b-4b35-8941-adbf2ab16f92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bc4f659ce964d4bb1c5b5ab9007c93a', 'request_id': 'req-84b8c7ef-032b-4b35-8941-adbf2ab16f92'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:19.756875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-84b8c7ef-032b-4b35-8941-adbf2ab16f92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bc4f659ce964d4bb1c5b5ab9007c93a', 'request_id': 'req-84b8c7ef-032b-4b35-8941-adbf2ab16f92'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:19.766081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-84b8c7ef-032b-4b35-8941-adbf2ab16f92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bc4f659ce964d4bb1c5b5ab9007c93a', 'request_id': 'req-84b8c7ef-032b-4b35-8941-adbf2ab16f92'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:19.778150 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-84b8c7ef-032b-4b35-8941-adbf2ab16f92 admin admin] The token cache is being invalidate because project 4bc4f659ce964d4bb1c5b5ab9007c93a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:19:19.778447 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-84b8c7ef-032b-4b35-8941-adbf2ab16f92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:19.779160 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-84b8c7ef-032b-4b35-8941-adbf2ab16f92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:19:19.781271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1190/2388] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:19:19 2022] DELETE /identity/v3/projects/4bc4f659ce964d4bb1c5b5ab9007c93a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:19.787106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314caa5b-4042-49ae-8bf8-b2b044310ce0 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.787475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314caa5b-4042-49ae-8bf8-b2b044310ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.787765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314caa5b-4042-49ae-8bf8-b2b044310ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.809438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-314caa5b-4042-49ae-8bf8-b2b044310ce0 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:19.810722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-314caa5b-4042-49ae-8bf8-b2b044310ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:19.831270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:19.879699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1199/2389] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:19.887258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932d28f-882c-466e-9ba5-c86977d7ce64 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.887556 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932d28f-882c-466e-9ba5-c86977d7ce64 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.887842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932d28f-882c-466e-9ba5-c86977d7ce64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.909570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a932d28f-882c-466e-9ba5-c86977d7ce64 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:19.911271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a932d28f-882c-466e-9ba5-c86977d7ce64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:19.922419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1191/2390] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:19.927309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314caa5b-4042-49ae-8bf8-b2b044310ce0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:19.937302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314caa5b-4042-49ae-8bf8-b2b044310ce0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:19.942093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcf293-19f6-4f06-8579-be4f34141875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:19.942649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcf293-19f6-4f06-8579-be4f34141875 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.942964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcf293-19f6-4f06-8579-be4f34141875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.943242 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcf293-19f6-4f06-8579-be4f34141875 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.943958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcf293-19f6-4f06-8579-be4f34141875 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:19:19.944272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcf293-19f6-4f06-8579-be4f34141875 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:19.945902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcf293-19f6-4f06-8579-be4f34141875 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:19.958690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1200/2391] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:19:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:19:19.963072 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932d28f-882c-466e-9ba5-c86977d7ce64 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:19.973080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932d28f-882c-466e-9ba5-c86977d7ce64 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:19.977956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da012b35-d27c-4b91-aad0-c4da50d961c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:19.978574 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da012b35-d27c-4b91-aad0-c4da50d961c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:19.978948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da012b35-d27c-4b91-aad0-c4da50d961c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:19.979229 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da012b35-d27c-4b91-aad0-c4da50d961c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:19.979915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da012b35-d27c-4b91-aad0-c4da50d961c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:19.981835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da012b35-d27c-4b91-aad0-c4da50d961c3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:20.009774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1192/2392] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:19:19 2022] POST /identity/v3/projects => generated 386 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:19:20.014116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcf293-19f6-4f06-8579-be4f34141875 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:20.024394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcf293-19f6-4f06-8579-be4f34141875 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:20.083119 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddcfe2-0d91-4878-b2ca-aabee96f4fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:20.083818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddcfe2-0d91-4878-b2ca-aabee96f4fcf admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:20.084023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddcfe2-0d91-4878-b2ca-aabee96f4fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:20.084302 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddcfe2-0d91-4878-b2ca-aabee96f4fcf admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:20.084978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddcfe2-0d91-4878-b2ca-aabee96f4fcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:20.086952 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ddcfe2-0d91-4878-b2ca-aabee96f4fcf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:20.110456 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1201/2393] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:19:20 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:20.116436 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da012b35-d27c-4b91-aad0-c4da50d961c3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:20.127178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da012b35-d27c-4b91-aad0-c4da50d961c3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:20.131888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdb7ab-146c-468b-b3d1-8e9e411b13f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:20.132442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdb7ab-146c-468b-b3d1-8e9e411b13f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:20.132727 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdb7ab-146c-468b-b3d1-8e9e411b13f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:20.133006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecdb7ab-146c-468b-b3d1-8e9e411b13f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:20.133712 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecdb7ab-146c-468b-b3d1-8e9e411b13f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:20.135371 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecdb7ab-146c-468b-b3d1-8e9e411b13f4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:20.146553 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1193/2394] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:19:20 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:20.150600 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddcfe2-0d91-4878-b2ca-aabee96f4fcf admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:20.162576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddcfe2-0d91-4878-b2ca-aabee96f4fcf admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:20.167390 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623d9e4-0b58-43da-9d90-e6ee9bcdd33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:20.168008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623d9e4-0b58-43da-9d90-e6ee9bcdd33c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:20.168287 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623d9e4-0b58-43da-9d90-e6ee9bcdd33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:20.168560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623d9e4-0b58-43da-9d90-e6ee9bcdd33c admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:20.169262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623d9e4-0b58-43da-9d90-e6ee9bcdd33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:20.170916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623d9e4-0b58-43da-9d90-e6ee9bcdd33c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:20.182155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1202/2395] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:19:20 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:19:20.186347 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdb7ab-146c-468b-b3d1-8e9e411b13f4 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:20.197124 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecdb7ab-146c-468b-b3d1-8e9e411b13f4 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:20.202013 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb642837-9aea-417c-a329-f66fb9ef753f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:20.203286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb642837-9aea-417c-a329-f66fb9ef753f admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:20.203575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb642837-9aea-417c-a329-f66fb9ef753f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:20.204112 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb642837-9aea-417c-a329-f66fb9ef753f admin admin] PATH_INFO: `/v3/projects/f9c87c7cbcb5481db04aaa2a494be32d/users/c95c223146c94cd29850744a208ed60d/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:20.220758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb642837-9aea-417c-a329-f66fb9ef753f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c87c7cbcb5481db04aaa2a494be32d, user_id=c95c223146c94cd29850744a208ed60d, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:20.223509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb642837-9aea-417c-a329-f66fb9ef753f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:20.237776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1194/2396] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:19:20 2022] PUT /identity/v3/projects/f9c87c7cbcb5481db04aaa2a494be32d/users/c95c223146c94cd29850744a208ed60d/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:20.243128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb41e0-03e8-48e1-a360-2413edb34ae5 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:20.243418 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb41e0-03e8-48e1-a360-2413edb34ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:20.243695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb41e0-03e8-48e1-a360-2413edb34ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:20.264399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-46fb41e0-03e8-48e1-a360-2413edb34ae5 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:20.266148 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-46fb41e0-03e8-48e1-a360-2413edb34ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:20.349195 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1203/2397] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:20.359286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb642837-9aea-417c-a329-f66fb9ef753f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:20.361053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fb642837-9aea-417c-a329-f66fb9ef753f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:20.390849 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:20.426687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb642837-9aea-417c-a329-f66fb9ef753f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:20.430507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de636167-6746-4b2d-8465-810bf6888e7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:20.431162 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de636167-6746-4b2d-8465-810bf6888e7c service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:20.431447 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de636167-6746-4b2d-8465-810bf6888e7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:20.431727 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de636167-6746-4b2d-8465-810bf6888e7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:20.441886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de636167-6746-4b2d-8465-810bf6888e7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:20.443372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de636167-6746-4b2d-8465-810bf6888e7c service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:20.453585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1195/2398] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:24.555950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb05d93-945f-4336-8d1a-4c0fead09708 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:24.556560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb05d93-945f-4336-8d1a-4c0fead09708 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:24.556560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb05d93-945f-4336-8d1a-4c0fead09708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:24.589981 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-abb05d93-945f-4336-8d1a-4c0fead09708 None None] MFA Rules not processed for user `c95c223146c94cd29850744a208ed60d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:24.591269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-abb05d93-945f-4336-8d1a-4c0fead09708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:24.610379 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:24.661865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1204/2399] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:19:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:24.666960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de636167-6746-4b2d-8465-810bf6888e7c service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:24.677289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de636167-6746-4b2d-8465-810bf6888e7c service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:24.682442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc98e5-3f5b-4c20-a78c-e6d58528b765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:24.683097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc98e5-3f5b-4c20-a78c-e6d58528b765 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:24.683382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc98e5-3f5b-4c20-a78c-e6d58528b765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:24.683660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc98e5-3f5b-4c20-a78c-e6d58528b765 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:24.684332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc98e5-3f5b-4c20-a78c-e6d58528b765 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:24.686266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc98e5-3f5b-4c20-a78c-e6d58528b765 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:24.733687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1196/2400] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:19:24 2022] POST /identity/v3/projects => generated 386 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:24.737891 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb05d93-945f-4336-8d1a-4c0fead09708 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:24.748932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb05d93-945f-4336-8d1a-4c0fead09708 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:24.812704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ba63e-0623-4a83-8f38-23725de7d344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:24.813413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ba63e-0623-4a83-8f38-23725de7d344 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:24.813684 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ba63e-0623-4a83-8f38-23725de7d344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:24.813963 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ba63e-0623-4a83-8f38-23725de7d344 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:24.815064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ba63e-0623-4a83-8f38-23725de7d344 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:24.817026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ba63e-0623-4a83-8f38-23725de7d344 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:24.846609 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1205/2401] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:19:24 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:19:24.850249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc98e5-3f5b-4c20-a78c-e6d58528b765 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:24.863060 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc98e5-3f5b-4c20-a78c-e6d58528b765 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:24.868342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ed377-538e-47ec-9644-bdd7e5a2b0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:24.869000 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ed377-538e-47ec-9644-bdd7e5a2b0ed admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:24.869316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ed377-538e-47ec-9644-bdd7e5a2b0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:24.869594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6ed377-538e-47ec-9644-bdd7e5a2b0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:24.870629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ed377-538e-47ec-9644-bdd7e5a2b0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:24.872686 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6ed377-538e-47ec-9644-bdd7e5a2b0ed admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:24.886339 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1197/2402] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:19:24 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:19:24.892357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ba63e-0623-4a83-8f38-23725de7d344 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:24.904655 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ba63e-0623-4a83-8f38-23725de7d344 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:24.911702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d764284-12bc-4886-a580-2a06e02087b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:24.912490 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d764284-12bc-4886-a580-2a06e02087b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:24.912699 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d764284-12bc-4886-a580-2a06e02087b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:24.912981 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d764284-12bc-4886-a580-2a06e02087b1 admin admin] PATH_INFO: `/v3/projects/0d4ea44092064483b49c1bcb418b8244/users/9ab7c715f5874a4bbe29990c1a31390f/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:24.937740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d764284-12bc-4886-a580-2a06e02087b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4ea44092064483b49c1bcb418b8244, user_id=9ab7c715f5874a4bbe29990c1a31390f, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:24.940555 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d764284-12bc-4886-a580-2a06e02087b1 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:24.957360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1206/2403] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:19:24 2022] PUT /identity/v3/projects/0d4ea44092064483b49c1bcb418b8244/users/9ab7c715f5874a4bbe29990c1a31390f/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:19:24.961582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ed377-538e-47ec-9644-bdd7e5a2b0ed admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:24.976473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6ed377-538e-47ec-9644-bdd7e5a2b0ed admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:25.063756 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5001593-d37b-4da1-8a2e-001066ba0d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:25.064403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5001593-d37b-4da1-8a2e-001066ba0d89 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:25.064922 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5001593-d37b-4da1-8a2e-001066ba0d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:25.065205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5001593-d37b-4da1-8a2e-001066ba0d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:25.066134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5001593-d37b-4da1-8a2e-001066ba0d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:25.068012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5001593-d37b-4da1-8a2e-001066ba0d89 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:25.080356 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1198/2404] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:19:24 2022] GET /identity/v3/roles => generated 1427 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:19:25.084726 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d764284-12bc-4886-a580-2a06e02087b1 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:25.096562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d764284-12bc-4886-a580-2a06e02087b1 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:25.102653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a4300-fc96-44a6-8056-c7443840c264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:25.103650 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a4300-fc96-44a6-8056-c7443840c264 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:25.103927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a4300-fc96-44a6-8056-c7443840c264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:25.104209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a4300-fc96-44a6-8056-c7443840c264 admin admin] PATH_INFO: `/v3/projects/0d4ea44092064483b49c1bcb418b8244/users/9ab7c715f5874a4bbe29990c1a31390f/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:25.106656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a4300-fc96-44a6-8056-c7443840c264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4ea44092064483b49c1bcb418b8244, user_id=9ab7c715f5874a4bbe29990c1a31390f, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:25.109559 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a4300-fc96-44a6-8056-c7443840c264 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:25.121838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1207/2405] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:19:25 2022] PUT /identity/v3/projects/0d4ea44092064483b49c1bcb418b8244/users/9ab7c715f5874a4bbe29990c1a31390f/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:19:27.877548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5001593-d37b-4da1-8a2e-001066ba0d89 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:27.880195 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f5001593-d37b-4da1-8a2e-001066ba0d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:28.074533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5001593-d37b-4da1-8a2e-001066ba0d89 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:28.185400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:28.208747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4f0a9-a338-4ffc-a0ff-a738c5130821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:28.209402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4f0a9-a338-4ffc-a0ff-a738c5130821 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:28.209739 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4f0a9-a338-4ffc-a0ff-a738c5130821 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:28.210009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc4f0a9-a338-4ffc-a0ff-a738c5130821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:28.212068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9bc4f0a9-a338-4ffc-a0ff-a738c5130821 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:28.381443 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4f0a9-a338-4ffc-a0ff-a738c5130821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:28.383513 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc4f0a9-a338-4ffc-a0ff-a738c5130821 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:28.397378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1199/2406] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:19:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:30.707563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4a198-07a0-48df-96f7-dac6b76a936a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.708124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4a198-07a0-48df-96f7-dac6b76a936a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.708124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4a198-07a0-48df-96f7-dac6b76a936a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:30.745973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-44f4a198-07a0-48df-96f7-dac6b76a936a None None] MFA Rules not processed for user `9ab7c715f5874a4bbe29990c1a31390f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:30.747220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-44f4a198-07a0-48df-96f7-dac6b76a936a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:30.863629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:30.874550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1208/2407] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:19:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:30.887096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4f0a9-a338-4ffc-a0ff-a738c5130821 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:30.888597 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9bc4f0a9-a338-4ffc-a0ff-a738c5130821 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:30.959649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc4f0a9-a338-4ffc-a0ff-a738c5130821 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:30.965411 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a091a8-e981-477e-bd1e-a1f1f703ec07 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e75edfc5bb404d63b27a355d5ab8af01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:30.966051 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a091a8-e981-477e-bd1e-a1f1f703ec07 service glance] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:30.966332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a091a8-e981-477e-bd1e-a1f1f703ec07 service glance] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:30.966611 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a091a8-e981-477e-bd1e-a1f1f703ec07 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:31.036736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a091a8-e981-477e-bd1e-a1f1f703ec07 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:31.038596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a091a8-e981-477e-bd1e-a1f1f703ec07 service glance] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:31.050861 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1200/2408] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:19:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:31.213938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465688d3-f618-4023-aa7e-b8229f826942 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:31.214479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465688d3-f618-4023-aa7e-b8229f826942 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:31.214973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465688d3-f618-4023-aa7e-b8229f826942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:31.238086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-465688d3-f618-4023-aa7e-b8229f826942 None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:31.239592 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-465688d3-f618-4023-aa7e-b8229f826942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:31.348691 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1209/2409] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:19:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:31.361300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a091a8-e981-477e-bd1e-a1f1f703ec07 service glance] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:31.362788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-52a091a8-e981-477e-bd1e-a1f1f703ec07 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:31.419448 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:31.450726 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a091a8-e981-477e-bd1e-a1f1f703ec07 service glance] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:31.456204 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a3ff9-308e-4ed0-9131-2291205a6334 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:31.456886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a3ff9-308e-4ed0-9131-2291205a6334 service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:31.457165 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a3ff9-308e-4ed0-9131-2291205a6334 service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:31.457443 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a3ff9-308e-4ed0-9131-2291205a6334 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:31.466842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a3ff9-308e-4ed0-9131-2291205a6334 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:31.469109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a3ff9-308e-4ed0-9131-2291205a6334 service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:31.479259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1201/2410] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:19:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:19:40.114959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d425191-ad5a-4862-98c8-b7c5d6c3dcdc None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:40.115677 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d425191-ad5a-4862-98c8-b7c5d6c3dcdc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:40.115677 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d425191-ad5a-4862-98c8-b7c5d6c3dcdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:40.135860 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4d425191-ad5a-4862-98c8-b7c5d6c3dcdc None None] MFA Rules not processed for user `c95c223146c94cd29850744a208ed60d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:40.137177 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4d425191-ad5a-4862-98c8-b7c5d6c3dcdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:40.144751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1210/2411] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:19:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:40.183146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a3ff9-308e-4ed0-9131-2291205a6334 service swift] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:40.193264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a3ff9-308e-4ed0-9131-2291205a6334 service swift] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:40.196970 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebfe11-8806-4638-9c26-de41b3f9f2c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e75edfc5bb404d63b27a355d5ab8af01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:40.197575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebfe11-8806-4638-9c26-de41b3f9f2c8 service glance] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:40.197860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebfe11-8806-4638-9c26-de41b3f9f2c8 service glance] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:40.198138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ebfe11-8806-4638-9c26-de41b3f9f2c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:40.199970 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-20ebfe11-8806-4638-9c26-de41b3f9f2c8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:40.216422 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ebfe11-8806-4638-9c26-de41b3f9f2c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:40.217965 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ebfe11-8806-4638-9c26-de41b3f9f2c8 service glance] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:40.232241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1202/2412] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:19:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:42.301847 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625300b5-8120-4638-bccb-1db1f8b1c920 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:42.302369 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625300b5-8120-4638-bccb-1db1f8b1c920 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:42.302493 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625300b5-8120-4638-bccb-1db1f8b1c920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:42.321667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-625300b5-8120-4638-bccb-1db1f8b1c920 None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:19:42.323106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-625300b5-8120-4638-bccb-1db1f8b1c920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:42.333338 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1211/2413] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:19:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:19:42.350083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebfe11-8806-4638-9c26-de41b3f9f2c8 service glance] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:42.367173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ebfe11-8806-4638-9c26-de41b3f9f2c8 service glance] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:42.373706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6352457-6660-4dec-b258-9567fc610c97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:42.374627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6352457-6660-4dec-b258-9567fc610c97 service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:42.375129 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6352457-6660-4dec-b258-9567fc610c97 service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:42.375562 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6352457-6660-4dec-b258-9567fc610c97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:42.389167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6352457-6660-4dec-b258-9567fc610c97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:42.390925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6352457-6660-4dec-b258-9567fc610c97 service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:42.400551 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1203/2414] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:19:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:19:48.350919 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625300b5-8120-4638-bccb-1db1f8b1c920 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:48.352369 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-625300b5-8120-4638-bccb-1db1f8b1c920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:48.411991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625300b5-8120-4638-bccb-1db1f8b1c920 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:48.415823 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b7b18-049b-418d-9989-870aabbf1729 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:48.416480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b7b18-049b-418d-9989-870aabbf1729 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:48.416781 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b7b18-049b-418d-9989-870aabbf1729 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:48.417057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b7b18-049b-418d-9989-870aabbf1729 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:48.419080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-895b7b18-049b-418d-9989-870aabbf1729 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:48.491614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b7b18-049b-418d-9989-870aabbf1729 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:48.493081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b7b18-049b-418d-9989-870aabbf1729 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:48.502497 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1212/2415] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:19:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:19:49.531696 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6352457-6660-4dec-b258-9567fc610c97 service swift] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:19:49.533115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d6352457-6660-4dec-b258-9567fc610c97 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:49.550085 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6352457-6660-4dec-b258-9567fc610c97 service swift] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:19:49.555406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a75b6-d33e-4e17-ab47-89cf7374ebf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:19:49.556030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a75b6-d33e-4e17-ab47-89cf7374ebf2 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:19:49.556349 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a75b6-d33e-4e17-ab47-89cf7374ebf2 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:19:49.556664 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a75b6-d33e-4e17-ab47-89cf7374ebf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:19:49.558544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-755a75b6-d33e-4e17-ab47-89cf7374ebf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:19:49.627261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a75b6-d33e-4e17-ab47-89cf7374ebf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:19:49.628873 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a75b6-d33e-4e17-ab47-89cf7374ebf2 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:19:49.637944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1204/2416] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:19:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:19:58.498413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=81 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:19:59.633706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=55 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:03.661417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b7b18-049b-418d-9989-870aabbf1729 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:03.662956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-895b7b18-049b-418d-9989-870aabbf1729 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:03.742593 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a75b6-d33e-4e17-ab47-89cf7374ebf2 service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:03.744649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-755a75b6-d33e-4e17-ab47-89cf7374ebf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:03.753493 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b7b18-049b-418d-9989-870aabbf1729 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:03.764120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa066c-deee-4fe0-b1a9-de466822a43e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:03.764893 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa066c-deee-4fe0-b1a9-de466822a43e service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:03.765288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa066c-deee-4fe0-b1a9-de466822a43e service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:03.765983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa066c-deee-4fe0-b1a9-de466822a43e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:03.768945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-09aa066c-deee-4fe0-b1a9-de466822a43e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:03.786216 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa066c-deee-4fe0-b1a9-de466822a43e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:03.787900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa066c-deee-4fe0-b1a9-de466822a43e service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:03.802362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1213/2417] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:20:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:03.841332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a75b6-d33e-4e17-ab47-89cf7374ebf2 service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:03.846976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ef89c-5589-4358-a71c-ac7dc3adefdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:03.847631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ef89c-5589-4358-a71c-ac7dc3adefdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:03.847915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ef89c-5589-4358-a71c-ac7dc3adefdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:03.848198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ef89c-5589-4358-a71c-ac7dc3adefdc admin admin] PATH_INFO: `/v3/users/b507df75a6564522bf6746587d081a03` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:03.849386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ef89c-5589-4358-a71c-ac7dc3adefdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b507df75a6564522bf6746587d081a03)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:03.851396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ef89c-5589-4358-a71c-ac7dc3adefdc admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:03.902705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f03ef89c-5589-4358-a71c-ac7dc3adefdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b507df75a6564522bf6746587d081a03'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.911118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f03ef89c-5589-4358-a71c-ac7dc3adefdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b507df75a6564522bf6746587d081a03'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.911841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f03ef89c-5589-4358-a71c-ac7dc3adefdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b507df75a6564522bf6746587d081a03'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.912626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f03ef89c-5589-4358-a71c-ac7dc3adefdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b507df75a6564522bf6746587d081a03'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.925237 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f03ef89c-5589-4358-a71c-ac7dc3adefdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b507df75a6564522bf6746587d081a03'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:03.940571 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1205/2418] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:20:03 2022] DELETE /identity/v3/users/b507df75a6564522bf6746587d081a03 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:20:04.127288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa066c-deee-4fe0-b1a9-de466822a43e service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:04.147872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa066c-deee-4fe0-b1a9-de466822a43e service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:04.227731 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:04.228404 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:04.228670 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:04.228946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] PATH_INFO: `/v3/projects/060a7c219492407581c39806cc5726a3` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:04.230157 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=060a7c219492407581c39806cc5726a3)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:04.235110 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:04.281669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '060a7c219492407581c39806cc5726a3', 'request_id': 'req-69a703b0-d04f-48ed-81a6-579375362c88'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:04.288652 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '060a7c219492407581c39806cc5726a3', 'request_id': 'req-69a703b0-d04f-48ed-81a6-579375362c88'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:04.294931 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '060a7c219492407581c39806cc5726a3', 'request_id': 'req-69a703b0-d04f-48ed-81a6-579375362c88'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:04.312005 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] The token cache is being invalidate because project 060a7c219492407581c39806cc5726a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:20:04.312295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:04.313034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:20:04.315183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1214/2419] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:20:04 2022] DELETE /identity/v3/projects/060a7c219492407581c39806cc5726a3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:20:13.936002 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:14.307736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:25.713222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ef89c-5589-4358-a71c-ac7dc3adefdc admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:25.714723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f03ef89c-5589-4358-a71c-ac7dc3adefdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:25.777211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ef89c-5589-4358-a71c-ac7dc3adefdc admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:25.829696 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b4779-70cb-44cc-b9a2-fea4d3f01ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:25.830318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b4779-70cb-44cc-b9a2-fea4d3f01ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:25.830596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b4779-70cb-44cc-b9a2-fea4d3f01ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:25.830899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b4779-70cb-44cc-b9a2-fea4d3f01ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:25.832673 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a10b4779-70cb-44cc-b9a2-fea4d3f01ee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:25.897785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b4779-70cb-44cc-b9a2-fea4d3f01ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:25.899277 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b4779-70cb-44cc-b9a2-fea4d3f01ee4 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:25.908832 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1206/2420] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:20:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:27.684942 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:27.686389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:27.797771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a703b0-d04f-48ed-81a6-579375362c88 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:27.804944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9cc8e-deaf-4281-9866-331181e9549a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:27.805653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9cc8e-deaf-4281-9866-331181e9549a service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:27.806002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9cc8e-deaf-4281-9866-331181e9549a service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:27.806363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9cc8e-deaf-4281-9866-331181e9549a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:27.808487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-48e9cc8e-deaf-4281-9866-331181e9549a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:27.825860 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9cc8e-deaf-4281-9866-331181e9549a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:27.828182 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e9cc8e-deaf-4281-9866-331181e9549a service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:27.843270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1215/2421] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:20:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:27.977025 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b4779-70cb-44cc-b9a2-fea4d3f01ee4 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:27.978459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a10b4779-70cb-44cc-b9a2-fea4d3f01ee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:27.995027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b4779-70cb-44cc-b9a2-fea4d3f01ee4 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:28.000704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb944cd4-1464-4da0-ad08-1ea1cdcb621f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:28.002471 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb944cd4-1464-4da0-ad08-1ea1cdcb621f service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:28.003407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb944cd4-1464-4da0-ad08-1ea1cdcb621f service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:28.003936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb944cd4-1464-4da0-ad08-1ea1cdcb621f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:28.006154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-cb944cd4-1464-4da0-ad08-1ea1cdcb621f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:28.006295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9cc8e-deaf-4281-9866-331181e9549a service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:28.012163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-48e9cc8e-deaf-4281-9866-331181e9549a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:28.019250 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb944cd4-1464-4da0-ad08-1ea1cdcb621f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:28.020889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb944cd4-1464-4da0-ad08-1ea1cdcb621f service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:28.035623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1207/2422] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:20:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:28.047781 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9cc8e-deaf-4281-9866-331181e9549a service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:28.055617 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361a567-5782-422b-811f-407da89e77cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:28.056329 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361a567-5782-422b-811f-407da89e77cc service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:28.056617 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361a567-5782-422b-811f-407da89e77cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:28.056918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361a567-5782-422b-811f-407da89e77cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:28.089138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3361a567-5782-422b-811f-407da89e77cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:28.092649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3361a567-5782-422b-811f-407da89e77cc service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:28.107527 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1216/2423] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:20:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:20:28.498451 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa61a97-ece7-4539-a00e-f4f83926f859 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:28.499202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa61a97-ece7-4539-a00e-f4f83926f859 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:28.499578 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa61a97-ece7-4539-a00e-f4f83926f859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:28.526918 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9aa61a97-ece7-4539-a00e-f4f83926f859 None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:28.528805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9aa61a97-ece7-4539-a00e-f4f83926f859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:28.626480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:28.635955 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1208/2424] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:20:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:20:28.664218 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361a567-5782-422b-811f-407da89e77cc service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:28.665665 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3361a567-5782-422b-811f-407da89e77cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:28.767946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361a567-5782-422b-811f-407da89e77cc service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:28.773526 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3330f2e-5109-42b0-a3de-038607c04ba8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:28.774303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3330f2e-5109-42b0-a3de-038607c04ba8 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:28.774640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3330f2e-5109-42b0-a3de-038607c04ba8 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:28.775011 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3330f2e-5109-42b0-a3de-038607c04ba8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:28.784675 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3330f2e-5109-42b0-a3de-038607c04ba8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:28.786405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3330f2e-5109-42b0-a3de-038607c04ba8 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:28.797564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1217/2425] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:20:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:20:29.952129 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa61a97-ece7-4539-a00e-f4f83926f859 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:29.973248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa61a97-ece7-4539-a00e-f4f83926f859 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:29.988027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e690ec-8eb7-4b75-8634-2066e1df07bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:29.989699 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e690ec-8eb7-4b75-8634-2066e1df07bf service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:29.990605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e690ec-8eb7-4b75-8634-2066e1df07bf service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:29.990928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e690ec-8eb7-4b75-8634-2066e1df07bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:29.993412 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-43e690ec-8eb7-4b75-8634-2066e1df07bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:30.051627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e690ec-8eb7-4b75-8634-2066e1df07bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:30.056220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e690ec-8eb7-4b75-8634-2066e1df07bf service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:30.089658 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1209/2426] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:20:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:38.793393 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=72 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:40.078965 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=7 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:20:43.406428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3330f2e-5109-42b0-a3de-038607c04ba8 service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:43.410039 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e3330f2e-5109-42b0-a3de-038607c04ba8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:43.482461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3330f2e-5109-42b0-a3de-038607c04ba8 service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:43.488228 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616ec71-5453-4692-b3e6-a795256d5540 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:43.488851 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616ec71-5453-4692-b3e6-a795256d5540 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:43.489134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616ec71-5453-4692-b3e6-a795256d5540 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:43.489411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616ec71-5453-4692-b3e6-a795256d5540 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:43.491328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2616ec71-5453-4692-b3e6-a795256d5540 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:43.504568 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616ec71-5453-4692-b3e6-a795256d5540 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:43.506183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616ec71-5453-4692-b3e6-a795256d5540 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:43.516875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1218/2427] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:20:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:45.671648 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e690ec-8eb7-4b75-8634-2066e1df07bf service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.711363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e690ec-8eb7-4b75-8634-2066e1df07bf service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:45.715645 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104f2c2-38b2-4812-b8d1-ab0272bffc0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:45.715834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104f2c2-38b2-4812-b8d1-ab0272bffc0a service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.716192 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104f2c2-38b2-4812-b8d1-ab0272bffc0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.716514 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104f2c2-38b2-4812-b8d1-ab0272bffc0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.718622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a104f2c2-38b2-4812-b8d1-ab0272bffc0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:45.742881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104f2c2-38b2-4812-b8d1-ab0272bffc0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:45.744335 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104f2c2-38b2-4812-b8d1-ab0272bffc0a service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:45.752542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1210/2428] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:20:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:45.882258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c405b9-145c-4bf2-83ac-45eb309b6acc None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.882909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c405b9-145c-4bf2-83ac-45eb309b6acc None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.882909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c405b9-145c-4bf2-83ac-45eb309b6acc None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.884267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1219/2429] 10.222.0.100 () {54 vars in 926 bytes} [Mon Aug 1 15:20:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:20:45.891206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f869d98-3a7d-4c48-b4f2-a1d7d20a002f None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:45.891558 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f869d98-3a7d-4c48-b4f2-a1d7d20a002f None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:45.891875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f869d98-3a7d-4c48-b4f2-a1d7d20a002f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:45.934941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4f869d98-3a7d-4c48-b4f2-a1d7d20a002f None None] MFA Rules not processed for user `ec50ac7c54324884a1b2a6c63d617907`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:45.936193 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4f869d98-3a7d-4c48-b4f2-a1d7d20a002f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:45.945785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1211/2430] 10.222.0.100 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:20:45.961541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c405b9-145c-4bf2-83ac-45eb309b6acc None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:45.963221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-03c405b9-145c-4bf2-83ac-45eb309b6acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:46.035795 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c405b9-145c-4bf2-83ac-45eb309b6acc None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:46.039328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd939c2-b8a4-4a80-acd3-3fbb8a511c3d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e75edfc5bb404d63b27a355d5ab8af01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:46.039974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd939c2-b8a4-4a80-acd3-3fbb8a511c3d service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.040258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd939c2-b8a4-4a80-acd3-3fbb8a511c3d service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.040532 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd939c2-b8a4-4a80-acd3-3fbb8a511c3d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.051287 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd939c2-b8a4-4a80-acd3-3fbb8a511c3d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:46.052644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd939c2-b8a4-4a80-acd3-3fbb8a511c3d service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:46.063739 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1220/2431] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:20:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:20:46.134178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39e3bb-d2fb-4da2-a861-50a3a6468bd4 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.134623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39e3bb-d2fb-4da2-a861-50a3a6468bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.135061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39e3bb-d2fb-4da2-a861-50a3a6468bd4 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.137255 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1212/2432] 10.222.0.100 () {54 vars in 926 bytes} [Mon Aug 1 15:20:46 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 01 15:20:46.142411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5eebdb-a410-4b5b-bfa8-4f5beee186e4 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.142737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5eebdb-a410-4b5b-bfa8-4f5beee186e4 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.143020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5eebdb-a410-4b5b-bfa8-4f5beee186e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.165890 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1c5eebdb-a410-4b5b-bfa8-4f5beee186e4 None None] MFA Rules not processed for user `e7f757eabcb84de6ab042e0ac5a8158c`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:46.168638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1c5eebdb-a410-4b5b-bfa8-4f5beee186e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:46.180131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1221/2433] 10.222.0.100 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:20:46.192092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39e3bb-d2fb-4da2-a861-50a3a6468bd4 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:46.203649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39e3bb-d2fb-4da2-a861-50a3a6468bd4 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:46.208382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c2b79-72e0-48d5-aab2-b7e91ff000ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:46.209211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c2b79-72e0-48d5-aab2-b7e91ff000ec service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.209689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c2b79-72e0-48d5-aab2-b7e91ff000ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.210133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c2b79-72e0-48d5-aab2-b7e91ff000ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.219296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c2b79-72e0-48d5-aab2-b7e91ff000ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:46.220783 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c2b79-72e0-48d5-aab2-b7e91ff000ec service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:46.229115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1213/2434] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:20:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:46.314885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1d5a1-64d4-4362-a2fc-3ac6b1f33ced None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.315178 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1d5a1-64d4-4362-a2fc-3ac6b1f33ced None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.315458 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1d5a1-64d4-4362-a2fc-3ac6b1f33ced None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.316839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1222/2435] 10.222.0.100 () {54 vars in 926 bytes} [Mon Aug 1 15:20:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:20:46.322719 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec80eb8-04ce-459a-b738-dd3590cb2a04 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.322982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec80eb8-04ce-459a-b738-dd3590cb2a04 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.323263 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec80eb8-04ce-459a-b738-dd3590cb2a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.344829 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4ec80eb8-04ce-459a-b738-dd3590cb2a04 None None] MFA Rules not processed for user `ec50ac7c54324884a1b2a6c63d617907`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:46.346070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4ec80eb8-04ce-459a-b738-dd3590cb2a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:46.356087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1214/2436] 10.222.0.100 () {58 vars in 1023 bytes} [Mon Aug 1 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:20:46.367521 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1d5a1-64d4-4362-a2fc-3ac6b1f33ced None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:46.380207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1d5a1-64d4-4362-a2fc-3ac6b1f33ced None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:46.383892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183d052-fb80-4edc-a2a5-e3a31f0ec9a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:46.384506 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183d052-fb80-4edc-a2a5-e3a31f0ec9a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.384790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183d052-fb80-4edc-a2a5-e3a31f0ec9a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.385066 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183d052-fb80-4edc-a2a5-e3a31f0ec9a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.395875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183d052-fb80-4edc-a2a5-e3a31f0ec9a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:46.397307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183d052-fb80-4edc-a2a5-e3a31f0ec9a6 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:46.408588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1223/2437] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:20:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:46.476677 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec80eb8-04ce-459a-b738-dd3590cb2a04 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:46.478087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4ec80eb8-04ce-459a-b738-dd3590cb2a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:46.550782 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec80eb8-04ce-459a-b738-dd3590cb2a04 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:46.564040 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b63df-4a9c-4c49-bc9b-039470cfc68e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:46.564604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b63df-4a9c-4c49-bc9b-039470cfc68e service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:46.564888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b63df-4a9c-4c49-bc9b-039470cfc68e service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:46.565165 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b63df-4a9c-4c49-bc9b-039470cfc68e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:46.567602 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-402b63df-4a9c-4c49-bc9b-039470cfc68e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:46.580457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b63df-4a9c-4c49-bc9b-039470cfc68e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:46.582063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402b63df-4a9c-4c49-bc9b-039470cfc68e service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:46.592594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1215/2438] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:20:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:49.246277 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5fc57-a216-4e77-8f3a-df7d5a85459b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:49.246776 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5fc57-a216-4e77-8f3a-df7d5a85459b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:49.246902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5fc57-a216-4e77-8f3a-df7d5a85459b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:49.266217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e2d5fc57-a216-4e77-8f3a-df7d5a85459b None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:20:49.267736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e2d5fc57-a216-4e77-8f3a-df7d5a85459b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:49.275479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1224/2439] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:20:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:20:49.288166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b63df-4a9c-4c49-bc9b-039470cfc68e service placement] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:49.301264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b63df-4a9c-4c49-bc9b-039470cfc68e service placement] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:49.306382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2150ea-d5c7-4ab8-b9b4-02315ef9d59a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:49.307036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2150ea-d5c7-4ab8-b9b4-02315ef9d59a service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:49.307325 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2150ea-d5c7-4ab8-b9b4-02315ef9d59a service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:49.307605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2150ea-d5c7-4ab8-b9b4-02315ef9d59a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:49.316561 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2150ea-d5c7-4ab8-b9b4-02315ef9d59a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:49.318087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2150ea-d5c7-4ab8-b9b4-02315ef9d59a service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:49.327022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1216/2440] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:20:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:20:55.871069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5fc57-a216-4e77-8f3a-df7d5a85459b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:20:55.885600 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5fc57-a216-4e77-8f3a-df7d5a85459b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:20:55.891461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5027e7c-9193-4360-8c36-b3fbf57eb9fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:20:55.892093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5027e7c-9193-4360-8c36-b3fbf57eb9fb service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:20:55.892373 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5027e7c-9193-4360-8c36-b3fbf57eb9fb service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:20:55.892644 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5027e7c-9193-4360-8c36-b3fbf57eb9fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:20:55.894571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d5027e7c-9193-4360-8c36-b3fbf57eb9fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:20:55.966325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5027e7c-9193-4360-8c36-b3fbf57eb9fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:20:55.968364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5027e7c-9193-4360-8c36-b3fbf57eb9fb service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:20:55.979432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1225/2441] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:20:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:20:59.324441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=81 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:05.973949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=94 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:21.103148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209535b-417a-420a-9906-7cc60fbe4da3 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:21.104224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209535b-417a-420a-9906-7cc60fbe4da3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:21.104369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7209535b-417a-420a-9906-7cc60fbe4da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:21.123715 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7209535b-417a-420a-9906-7cc60fbe4da3 None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:21.124939 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7209535b-417a-420a-9906-7cc60fbe4da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:21.132146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1217/2442] 10.222.0.100 () {58 vars in 972 bytes} [Mon Aug 1 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:21.144361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5027e7c-9193-4360-8c36-b3fbf57eb9fb service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:21.156804 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5027e7c-9193-4360-8c36-b3fbf57eb9fb service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:21.162062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143c4c2-f987-4910-868b-94fbc55187ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:21.162715 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143c4c2-f987-4910-868b-94fbc55187ba service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:21.163002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143c4c2-f987-4910-868b-94fbc55187ba service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:21.163281 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143c4c2-f987-4910-868b-94fbc55187ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:21.175294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143c4c2-f987-4910-868b-94fbc55187ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:21.176908 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143c4c2-f987-4910-868b-94fbc55187ba service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:21.185853 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1226/2443] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:21:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:21:31.118626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:31.182330 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:36.321706 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c3bea-7b61-4afe-8c54-a0d7fea4448d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:36.322343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c3bea-7b61-4afe-8c54-a0d7fea4448d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:36.322343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c3bea-7b61-4afe-8c54-a0d7fea4448d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:36.343016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-620c3bea-7b61-4afe-8c54-a0d7fea4448d None None] MFA Rules not processed for user `ca1d32ee23bc42b38ea9a043f7a850f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:36.344497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-620c3bea-7b61-4afe-8c54-a0d7fea4448d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:36.352390 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1218/2444] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:21:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:21:46.337428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:21:56.519539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143c4c2-f987-4910-868b-94fbc55187ba service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:56.521286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a143c4c2-f987-4910-868b-94fbc55187ba service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:56.593885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c3bea-7b61-4afe-8c54-a0d7fea4448d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:21:56.628850 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c3bea-7b61-4afe-8c54-a0d7fea4448d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:56.646531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8d2cc-48c7-499a-a40e-eea60e918d58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:56.647292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8d2cc-48c7-499a-a40e-eea60e918d58 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:56.647639 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8d2cc-48c7-499a-a40e-eea60e918d58 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:56.647981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8d2cc-48c7-499a-a40e-eea60e918d58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:56.655068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-93f8d2cc-48c7-499a-a40e-eea60e918d58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:56.660806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143c4c2-f987-4910-868b-94fbc55187ba service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:21:56.672895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4e7b5-513b-4821-8510-6a19434f3ea8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:21:56.673629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e7b5-513b-4821-8510-6a19434f3ea8 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:56.673977 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e7b5-513b-4821-8510-6a19434f3ea8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:56.674315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4e7b5-513b-4821-8510-6a19434f3ea8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:56.677198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f2e4e7b5-513b-4821-8510-6a19434f3ea8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:56.784340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8d2cc-48c7-499a-a40e-eea60e918d58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:56.787286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8d2cc-48c7-499a-a40e-eea60e918d58 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:56.799711 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1219/2445] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:21:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:21:56.839779 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4e7b5-513b-4821-8510-6a19434f3ea8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:21:56.841552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4e7b5-513b-4821-8510-6a19434f3ea8 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:21:56.874953 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1227/2446] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:21:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:21:59.158431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2da5-81db-49df-9143-714708abff7c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:21:59.159008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2da5-81db-49df-9143-714708abff7c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:21:59.159103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e2da5-81db-49df-9143-714708abff7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:21:59.181692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-c82e2da5-81db-49df-9143-714708abff7c None None] MFA Rules not processed for user `c6795a827b8d411aaa165db7b38435ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:21:59.183004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c82e2da5-81db-49df-9143-714708abff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:21:59.191081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1220/2447] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:21:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:22:06.853666 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=57 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:09.175475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=31 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:22:48.469003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841724b-7228-4884-bd72-69b893858456 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:48.469587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841724b-7228-4884-bd72-69b893858456 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:48.469587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841724b-7228-4884-bd72-69b893858456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:48.489553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e841724b-7228-4884-bd72-69b893858456 None None] MFA Rules not processed for user `424ccf230e16423a8dd32097d90c4995`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:22:48.491359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e841724b-7228-4884-bd72-69b893858456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:48.555478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1228/2448] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:22:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:22:53.790705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e2da5-81db-49df-9143-714708abff7c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:22:53.792218 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c82e2da5-81db-49df-9143-714708abff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:53.831410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e2da5-81db-49df-9143-714708abff7c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:22:53.838269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b909a24-c759-4424-9c20-860fea06d64f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:22:53.838919 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b909a24-c759-4424-9c20-860fea06d64f service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:22:53.839203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b909a24-c759-4424-9c20-860fea06d64f service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:22:53.839480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b909a24-c759-4424-9c20-860fea06d64f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:22:53.842401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4b909a24-c759-4424-9c20-860fea06d64f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:22:53.857518 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b909a24-c759-4424-9c20-860fea06d64f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:22:53.859163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b909a24-c759-4424-9c20-860fea06d64f service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:22:53.870168 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1221/2449] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:22:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:22:58.545272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=25 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:03.865649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=15 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:05.247566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841724b-7228-4884-bd72-69b893858456 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:05.257908 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841724b-7228-4884-bd72-69b893858456 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:05.261422 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61703577-4380-4464-ada2-13ad7422d04c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:05.262037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61703577-4380-4464-ada2-13ad7422d04c service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:05.262320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61703577-4380-4464-ada2-13ad7422d04c service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:05.262599 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61703577-4380-4464-ada2-13ad7422d04c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:05.264986 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-61703577-4380-4464-ada2-13ad7422d04c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:05.352075 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61703577-4380-4464-ada2-13ad7422d04c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:05.353704 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61703577-4380-4464-ada2-13ad7422d04c service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:05.373898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1229/2450] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:09.636891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b909a24-c759-4424-9c20-860fea06d64f service placement] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:09.653785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b909a24-c759-4424-9c20-860fea06d64f service placement] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:09.668929 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa7f8a-8de6-4f65-b587-9b68c33cfff0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:09.670536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa7f8a-8de6-4f65-b587-9b68c33cfff0 service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:09.671103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa7f8a-8de6-4f65-b587-9b68c33cfff0 service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:09.671867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa7f8a-8de6-4f65-b587-9b68c33cfff0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:09.675895 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-74fa7f8a-8de6-4f65-b587-9b68c33cfff0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:09.707082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa7f8a-8de6-4f65-b587-9b68c33cfff0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:09.709752 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fa7f8a-8de6-4f65-b587-9b68c33cfff0 service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:09.728504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1222/2451] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:23:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:23:09.731435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61703577-4380-4464-ada2-13ad7422d04c service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:09.733864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-61703577-4380-4464-ada2-13ad7422d04c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:09.753291 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61703577-4380-4464-ada2-13ad7422d04c service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:09.767875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a67eb1-dbe7-40b8-83e4-90572bc72014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:09.769874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a67eb1-dbe7-40b8-83e4-90572bc72014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:09.770181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a67eb1-dbe7-40b8-83e4-90572bc72014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:09.770655 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a67eb1-dbe7-40b8-83e4-90572bc72014 admin admin] PATH_INFO: `/v3/users/ca1d32ee23bc42b38ea9a043f7a850f5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:09.772303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a67eb1-dbe7-40b8-83e4-90572bc72014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca1d32ee23bc42b38ea9a043f7a850f5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:09.776146 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a67eb1-dbe7-40b8-83e4-90572bc72014 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:09.835402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d2a67eb1-dbe7-40b8-83e4-90572bc72014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca1d32ee23bc42b38ea9a043f7a850f5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:09.836231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d2a67eb1-dbe7-40b8-83e4-90572bc72014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca1d32ee23bc42b38ea9a043f7a850f5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:09.844842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d2a67eb1-dbe7-40b8-83e4-90572bc72014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca1d32ee23bc42b38ea9a043f7a850f5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:09.845613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d2a67eb1-dbe7-40b8-83e4-90572bc72014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca1d32ee23bc42b38ea9a043f7a850f5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:09.860116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d2a67eb1-dbe7-40b8-83e4-90572bc72014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca1d32ee23bc42b38ea9a043f7a850f5'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:09.883224 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1230/2452] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:23:09 2022] DELETE /identity/v3/users/ca1d32ee23bc42b38ea9a043f7a850f5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:10.072948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa7f8a-8de6-4f65-b587-9b68c33cfff0 service placement] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:10.088653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa7f8a-8de6-4f65-b587-9b68c33cfff0 service placement] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:10.158786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:10.159451 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:10.159767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:10.160047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77 admin admin] PATH_INFO: `/v3/projects/e5e42e0f0ef74615ac27486ada272fd8` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:10.161798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e42e0f0ef74615ac27486ada272fd8)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:10.165113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:10.226030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e42e0f0ef74615ac27486ada272fd8', 'request_id': 'req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:10.234940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e42e0f0ef74615ac27486ada272fd8', 'request_id': 'req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:10.248564 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e42e0f0ef74615ac27486ada272fd8', 'request_id': 'req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:10.265006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77 admin admin] The token cache is being invalidate because project e5e42e0f0ef74615ac27486ada272fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:23:10.265325 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:10.265978 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-c4e8f98b-77eb-45cf-a54d-97e5076bfd77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:10.267694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1223/2453] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:23:10 2022] DELETE /identity/v3/projects/e5e42e0f0ef74615ac27486ada272fd8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:10.274221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d90279-4d2f-43d8-8400-461e1fe53d07 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:10.274221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d90279-4d2f-43d8-8400-461e1fe53d07 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:10.274221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d90279-4d2f-43d8-8400-461e1fe53d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:10.328191 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a5d90279-4d2f-43d8-8400-461e1fe53d07 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:10.329410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a5d90279-4d2f-43d8-8400-461e1fe53d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:10.339522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:10.393592 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1231/2454] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:10.401791 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb08da-5cff-466e-b340-36b5bd6794f6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:10.402087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb08da-5cff-466e-b340-36b5bd6794f6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:10.402369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb08da-5cff-466e-b340-36b5bd6794f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:10.425136 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-31cb08da-5cff-466e-b340-36b5bd6794f6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:10.426434 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-31cb08da-5cff-466e-b340-36b5bd6794f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:10.442130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1224/2455] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:10.446463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d90279-4d2f-43d8-8400-461e1fe53d07 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:10.456944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d90279-4d2f-43d8-8400-461e1fe53d07 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:10.462196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61868f-4943-4b8b-bb8c-8983b381441e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:10.462926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61868f-4943-4b8b-bb8c-8983b381441e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:10.463270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61868f-4943-4b8b-bb8c-8983b381441e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:10.463610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61868f-4943-4b8b-bb8c-8983b381441e admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:10.464365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61868f-4943-4b8b-bb8c-8983b381441e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:23:10.464735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61868f-4943-4b8b-bb8c-8983b381441e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:10.466385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb61868f-4943-4b8b-bb8c-8983b381441e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:10.479201 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1232/2456] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:23:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:23:10.483560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb08da-5cff-466e-b340-36b5bd6794f6 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:10.494135 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb08da-5cff-466e-b340-36b5bd6794f6 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:10.498997 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32321bae-a1c1-4327-bbbb-825fd0f8cad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:10.499693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32321bae-a1c1-4327-bbbb-825fd0f8cad2 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:10.499971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32321bae-a1c1-4327-bbbb-825fd0f8cad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:10.500267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32321bae-a1c1-4327-bbbb-825fd0f8cad2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:10.500936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32321bae-a1c1-4327-bbbb-825fd0f8cad2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:10.503024 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32321bae-a1c1-4327-bbbb-825fd0f8cad2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:10.544598 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1225/2457] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:23:10 2022] POST /identity/v3/projects => generated 384 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:10.549109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61868f-4943-4b8b-bb8c-8983b381441e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:10.559976 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61868f-4943-4b8b-bb8c-8983b381441e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:10.623913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77226d-969e-46b8-8514-5131086be899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:10.624532 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77226d-969e-46b8-8514-5131086be899 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:10.624811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77226d-969e-46b8-8514-5131086be899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:10.625100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a77226d-969e-46b8-8514-5131086be899 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:10.625745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77226d-969e-46b8-8514-5131086be899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:10.627682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a77226d-969e-46b8-8514-5131086be899 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:10.648724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1233/2458] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:23:10 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:23:10.653218 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32321bae-a1c1-4327-bbbb-825fd0f8cad2 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:10.664318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32321bae-a1c1-4327-bbbb-825fd0f8cad2 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:10.669054 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c4ecf-2b74-426b-a064-f65880777efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:10.669664 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c4ecf-2b74-426b-a064-f65880777efb admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:10.669953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c4ecf-2b74-426b-a064-f65880777efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:10.670231 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c4ecf-2b74-426b-a064-f65880777efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:10.670967 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c4ecf-2b74-426b-a064-f65880777efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:10.672563 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c4ecf-2b74-426b-a064-f65880777efb admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:10.682467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1226/2459] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:23:10 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:10.686415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77226d-969e-46b8-8514-5131086be899 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:10.697054 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a77226d-969e-46b8-8514-5131086be899 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:10.702074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51664708-479b-4229-beb7-be3f791debb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:10.702721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51664708-479b-4229-beb7-be3f791debb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:10.703023 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51664708-479b-4229-beb7-be3f791debb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:10.703300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51664708-479b-4229-beb7-be3f791debb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:10.704033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51664708-479b-4229-beb7-be3f791debb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:10.705628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51664708-479b-4229-beb7-be3f791debb3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:10.716892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1234/2460] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:23:10 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:23:10.720893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c4ecf-2b74-426b-a064-f65880777efb admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:10.730679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c4ecf-2b74-426b-a064-f65880777efb admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:10.735344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830df50d-0e21-4e1c-85a6-45cc7072c01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:10.736090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df50d-0e21-4e1c-85a6-45cc7072c01e admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:10.736380 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df50d-0e21-4e1c-85a6-45cc7072c01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:10.736671 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df50d-0e21-4e1c-85a6-45cc7072c01e admin admin] PATH_INFO: `/v3/projects/89ef702cd1114e3eac72ff64e2d80b1c/users/49f98078ac424b1284932839be652265/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:10.751425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830df50d-0e21-4e1c-85a6-45cc7072c01e admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ef702cd1114e3eac72ff64e2d80b1c, user_id=49f98078ac424b1284932839be652265, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:10.754047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830df50d-0e21-4e1c-85a6-45cc7072c01e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:10.767721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1227/2461] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:23:10 2022] PUT /identity/v3/projects/89ef702cd1114e3eac72ff64e2d80b1c/users/49f98078ac424b1284932839be652265/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:10.773370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d24283-4296-48ce-aedf-2ad426eed0df None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:10.773879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d24283-4296-48ce-aedf-2ad426eed0df None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:10.773980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d24283-4296-48ce-aedf-2ad426eed0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:10.792547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-51d24283-4296-48ce-aedf-2ad426eed0df None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:10.793755 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-51d24283-4296-48ce-aedf-2ad426eed0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:10.853576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1235/2462] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:10.863115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830df50d-0e21-4e1c-85a6-45cc7072c01e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:10.864887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-830df50d-0e21-4e1c-85a6-45cc7072c01e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:10.871494 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:10.922830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830df50d-0e21-4e1c-85a6-45cc7072c01e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:10.926506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac3cd3-716d-4dd5-9531-7b17ba02513f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:10.927147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3cd3-716d-4dd5-9531-7b17ba02513f service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:10.927534 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3cd3-716d-4dd5-9531-7b17ba02513f service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:10.927814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac3cd3-716d-4dd5-9531-7b17ba02513f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:10.936996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac3cd3-716d-4dd5-9531-7b17ba02513f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:10.938446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac3cd3-716d-4dd5-9531-7b17ba02513f service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:10.948241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1228/2463] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:14.890777 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd32596-7102-4c59-a938-5890fb70be88 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:14.891328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd32596-7102-4c59-a938-5890fb70be88 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:14.891328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd32596-7102-4c59-a938-5890fb70be88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:14.923668 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4bd32596-7102-4c59-a938-5890fb70be88 None None] MFA Rules not processed for user `49f98078ac424b1284932839be652265`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:14.924917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4bd32596-7102-4c59-a938-5890fb70be88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:14.934499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:14.994189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1236/2464] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:23:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:15.017329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac3cd3-716d-4dd5-9531-7b17ba02513f service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:15.019380 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fcac3cd3-716d-4dd5-9531-7b17ba02513f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:15.141935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac3cd3-716d-4dd5-9531-7b17ba02513f service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:15.148510 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2bd9b-db10-470b-ac99-cea961f1cd5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:15.149154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2bd9b-db10-470b-ac99-cea961f1cd5b service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:15.149441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2bd9b-db10-470b-ac99-cea961f1cd5b service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:15.149718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2bd9b-db10-470b-ac99-cea961f1cd5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:15.158488 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2bd9b-db10-470b-ac99-cea961f1cd5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:15.160155 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2bd9b-db10-470b-ac99-cea961f1cd5b service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:15.171590 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1229/2465] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:23:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:16.509777 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f74af-7cec-48ed-969c-49d7f29e6d2f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:16.510275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f74af-7cec-48ed-969c-49d7f29e6d2f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:16.510377 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f74af-7cec-48ed-969c-49d7f29e6d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:16.530784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-055f74af-7cec-48ed-969c-49d7f29e6d2f None None] MFA Rules not processed for user `49f98078ac424b1284932839be652265`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:16.532006 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-055f74af-7cec-48ed-969c-49d7f29e6d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:16.539947 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1237/2466] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:23:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:16.851830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2bd9b-db10-470b-ac99-cea961f1cd5b service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:16.853277 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a5b2bd9b-db10-470b-ac99-cea961f1cd5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:16.865538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2bd9b-db10-470b-ac99-cea961f1cd5b service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:16.869200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f8cd-34cd-41a8-8856-81b49fe8721e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:16.870009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f8cd-34cd-41a8-8856-81b49fe8721e service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:16.870289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f8cd-34cd-41a8-8856-81b49fe8721e service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:16.870561 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216f8cd-34cd-41a8-8856-81b49fe8721e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:16.872304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7216f8cd-34cd-41a8-8856-81b49fe8721e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:16.882467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216f8cd-34cd-41a8-8856-81b49fe8721e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:16.883925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216f8cd-34cd-41a8-8856-81b49fe8721e service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:16.892066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1230/2467] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:23:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:23:26.525446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=25 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:26.888460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=76 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:34.785997 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836cc08-1a7d-4508-b42f-0291689ab122 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:34.786469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836cc08-1a7d-4508-b42f-0291689ab122 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:34.786577 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836cc08-1a7d-4508-b42f-0291689ab122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:34.815391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f836cc08-1a7d-4508-b42f-0291689ab122 None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:34.816804 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f836cc08-1a7d-4508-b42f-0291689ab122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:34.883453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1238/2468] 10.222.0.100 () {58 vars in 972 bytes} [Mon Aug 1 15:23:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:34.947904 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f8cd-34cd-41a8-8856-81b49fe8721e service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:34.949416 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7216f8cd-34cd-41a8-8856-81b49fe8721e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:35.031166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216f8cd-34cd-41a8-8856-81b49fe8721e service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:35.036322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e6e04-69d3-4cc3-aa59-9643b1a4ac2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:35.036963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e6e04-69d3-4cc3-aa59-9643b1a4ac2d service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:35.037247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e6e04-69d3-4cc3-aa59-9643b1a4ac2d service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:35.038213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324e6e04-69d3-4cc3-aa59-9643b1a4ac2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:35.052264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e6e04-69d3-4cc3-aa59-9643b1a4ac2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:35.054243 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324e6e04-69d3-4cc3-aa59-9643b1a4ac2d service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:35.065569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1231/2469] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:23:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:23:36.853631 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547bfa4-3bfd-49d5-8c13-e582ab86210d None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:36.854161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547bfa4-3bfd-49d5-8c13-e582ab86210d None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:36.854758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547bfa4-3bfd-49d5-8c13-e582ab86210d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:36.880810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7547bfa4-3bfd-49d5-8c13-e582ab86210d None None] MFA Rules not processed for user `f33ea26c1f72445bb4cb50ea503600dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:23:36.882221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7547bfa4-3bfd-49d5-8c13-e582ab86210d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:36.938322 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1239/2470] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:23:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:23:42.518872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e6e04-69d3-4cc3-aa59-9643b1a4ac2d service swift] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.520343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-324e6e04-69d3-4cc3-aa59-9643b1a4ac2d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:42.535293 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324e6e04-69d3-4cc3-aa59-9643b1a4ac2d service swift] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.541911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26c541-7d61-4f4b-9507-27aa7b3d085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.542752 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26c541-7d61-4f4b-9507-27aa7b3d085d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.543260 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26c541-7d61-4f4b-9507-27aa7b3d085d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.543767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26c541-7d61-4f4b-9507-27aa7b3d085d admin admin] PATH_INFO: `/v3/users/c95c223146c94cd29850744a208ed60d` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.545440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26c541-7d61-4f4b-9507-27aa7b3d085d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c95c223146c94cd29850744a208ed60d)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.548131 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26c541-7d61-4f4b-9507-27aa7b3d085d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.597059 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-fc26c541-7d61-4f4b-9507-27aa7b3d085d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c95c223146c94cd29850744a208ed60d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:42.605030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-fc26c541-7d61-4f4b-9507-27aa7b3d085d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c95c223146c94cd29850744a208ed60d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:42.605659 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-fc26c541-7d61-4f4b-9507-27aa7b3d085d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c95c223146c94cd29850744a208ed60d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:42.606261 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-fc26c541-7d61-4f4b-9507-27aa7b3d085d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c95c223146c94cd29850744a208ed60d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:42.618317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-fc26c541-7d61-4f4b-9507-27aa7b3d085d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c95c223146c94cd29850744a208ed60d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:42.630394 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1232/2471] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:23:42 2022] DELETE /identity/v3/users/c95c223146c94cd29850744a208ed60d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:42.818232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547bfa4-3bfd-49d5-8c13-e582ab86210d None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.832262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547bfa4-3bfd-49d5-8c13-e582ab86210d None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:42.894162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:42.894849 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:42.895136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:42.895416 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] PATH_INFO: `/v3/projects/f9c87c7cbcb5481db04aaa2a494be32d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:42.896594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c87c7cbcb5481db04aaa2a494be32d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:42.899206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:42.945520 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c87c7cbcb5481db04aaa2a494be32d', 'request_id': 'req-93d21073-cc08-40c4-b59a-9117cbedb42d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:42.947426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:42.954152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c87c7cbcb5481db04aaa2a494be32d', 'request_id': 'req-93d21073-cc08-40c4-b59a-9117cbedb42d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:42.960714 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c87c7cbcb5481db04aaa2a494be32d', 'request_id': 'req-93d21073-cc08-40c4-b59a-9117cbedb42d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:42.972925 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] The token cache is being invalidate because project f9c87c7cbcb5481db04aaa2a494be32d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:23:42.973222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:42.973842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:42.975529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1240/2472] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:23:42 2022] DELETE /identity/v3/projects/f9c87c7cbcb5481db04aaa2a494be32d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:42.979619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26c541-7d61-4f4b-9507-27aa7b3d085d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:42.981171 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fc26c541-7d61-4f4b-9507-27aa7b3d085d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:42.994439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26c541-7d61-4f4b-9507-27aa7b3d085d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:43.050393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01225619-13e8-4e91-96da-25c76b88022f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:43.051061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01225619-13e8-4e91-96da-25c76b88022f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:43.051351 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01225619-13e8-4e91-96da-25c76b88022f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:43.051631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01225619-13e8-4e91-96da-25c76b88022f admin admin] PATH_INFO: `/v3/users/9ab7c715f5874a4bbe29990c1a31390f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:43.052826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01225619-13e8-4e91-96da-25c76b88022f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab7c715f5874a4bbe29990c1a31390f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:43.054810 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01225619-13e8-4e91-96da-25c76b88022f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:43.057092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:43.102894 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-01225619-13e8-4e91-96da-25c76b88022f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab7c715f5874a4bbe29990c1a31390f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.110346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-01225619-13e8-4e91-96da-25c76b88022f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab7c715f5874a4bbe29990c1a31390f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.110990 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-01225619-13e8-4e91-96da-25c76b88022f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab7c715f5874a4bbe29990c1a31390f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.111653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-01225619-13e8-4e91-96da-25c76b88022f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab7c715f5874a4bbe29990c1a31390f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.123677 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-01225619-13e8-4e91-96da-25c76b88022f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab7c715f5874a4bbe29990c1a31390f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.136203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1233/2473] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:23:42 2022] DELETE /identity/v3/users/9ab7c715f5874a4bbe29990c1a31390f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:43.308552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:23:43.310005 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:23:43.324117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d21073-cc08-40c4-b59a-9117cbedb42d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:23:43.382065 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:23:43.382737 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:23:43.383025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:23:43.383304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] PATH_INFO: `/v3/projects/0d4ea44092064483b49c1bcb418b8244` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:23:43.384482 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d4ea44092064483b49c1bcb418b8244)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:23:43.386415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:23:43.432547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d4ea44092064483b49c1bcb418b8244', 'request_id': 'req-6967fb2d-6564-4788-a004-19a4647a506e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.438937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d4ea44092064483b49c1bcb418b8244', 'request_id': 'req-6967fb2d-6564-4788-a004-19a4647a506e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.445004 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d4ea44092064483b49c1bcb418b8244', 'request_id': 'req-6967fb2d-6564-4788-a004-19a4647a506e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.458294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] The token cache is being invalidate because project 0d4ea44092064483b49c1bcb418b8244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:23:43.458581 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.459266 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:23:43.460793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1241/2474] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:23:43 2022] DELETE /identity/v3/projects/0d4ea44092064483b49c1bcb418b8244 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:23:53.132714 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:23:53.455263 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:45.956403 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344a344-71cc-40a3-8e5e-48eb23c68bcf None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:45.957038 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344a344-71cc-40a3-8e5e-48eb23c68bcf None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:45.957038 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344a344-71cc-40a3-8e5e-48eb23c68bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:45.976673 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4344a344-71cc-40a3-8e5e-48eb23c68bcf None None] MFA Rules not processed for user `c6795a827b8d411aaa165db7b38435ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:24:45.977936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4344a344-71cc-40a3-8e5e-48eb23c68bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:46.038123 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1234/2475] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:24:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:24:46.080261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:46.082001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:46.148279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967fb2d-6564-4788-a004-19a4647a506e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:46.152092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56251a88-37a2-4dfb-b503-095ba5b66f4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:46.152711 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56251a88-37a2-4dfb-b503-095ba5b66f4b service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:46.152990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56251a88-37a2-4dfb-b503-095ba5b66f4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:46.153266 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56251a88-37a2-4dfb-b503-095ba5b66f4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:46.155052 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-56251a88-37a2-4dfb-b503-095ba5b66f4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:46.226331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56251a88-37a2-4dfb-b503-095ba5b66f4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:46.227824 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56251a88-37a2-4dfb-b503-095ba5b66f4b service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:46.240345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1242/2476] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:24:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:47.005313 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344a344-71cc-40a3-8e5e-48eb23c68bcf None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:47.006912 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4344a344-71cc-40a3-8e5e-48eb23c68bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:47.066043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344a344-71cc-40a3-8e5e-48eb23c68bcf None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:47.070116 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a521f-9a1b-4a3a-96fd-321a1e88e3d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:47.070811 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a521f-9a1b-4a3a-96fd-321a1e88e3d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:47.071099 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a521f-9a1b-4a3a-96fd-321a1e88e3d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:47.071383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a521f-9a1b-4a3a-96fd-321a1e88e3d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:47.073241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-339a521f-9a1b-4a3a-96fd-321a1e88e3d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:47.088541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a521f-9a1b-4a3a-96fd-321a1e88e3d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:47.089992 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a521f-9a1b-4a3a-96fd-321a1e88e3d9 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:47.102964 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1235/2477] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:24:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:48.791942 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56251a88-37a2-4dfb-b503-095ba5b66f4b service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:24:48.793295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-56251a88-37a2-4dfb-b503-095ba5b66f4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:48.807300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56251a88-37a2-4dfb-b503-095ba5b66f4b service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:24:48.812836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc726c5-2594-4afb-bc26-0757ff254b3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:24:48.813447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc726c5-2594-4afb-bc26-0757ff254b3a service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:24:48.813727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc726c5-2594-4afb-bc26-0757ff254b3a service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:24:48.814025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc726c5-2594-4afb-bc26-0757ff254b3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:24:48.816063 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bcc726c5-2594-4afb-bc26-0757ff254b3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:24:48.921093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc726c5-2594-4afb-bc26-0757ff254b3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:24:48.923090 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc726c5-2594-4afb-bc26-0757ff254b3a service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:24:48.945648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1243/2478] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:24:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:24:57.096370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=51 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:24:58.942567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:06.127201 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a521f-9a1b-4a3a-96fd-321a1e88e3d9 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:06.136545 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a521f-9a1b-4a3a-96fd-321a1e88e3d9 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:06.140751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df6700-495e-4ce1-b006-df069ca36f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:06.141353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df6700-495e-4ce1-b006-df069ca36f5e service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:06.141632 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df6700-495e-4ce1-b006-df069ca36f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:06.141912 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df6700-495e-4ce1-b006-df069ca36f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:06.143687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a6df6700-495e-4ce1-b006-df069ca36f5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:06.214101 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df6700-495e-4ce1-b006-df069ca36f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:06.215572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df6700-495e-4ce1-b006-df069ca36f5e service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:06.225878 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1236/2479] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:25:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:07.890881 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4d9a6-25b2-4ba4-b953-980984fd5eb7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:07.891606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4d9a6-25b2-4ba4-b953-980984fd5eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:07.891606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4d9a6-25b2-4ba4-b953-980984fd5eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:07.951250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3dc4d9a6-25b2-4ba4-b953-980984fd5eb7 None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:07.952481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3dc4d9a6-25b2-4ba4-b953-980984fd5eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:08.062904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1244/2480] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:25:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:08.144127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df6700-495e-4ce1-b006-df069ca36f5e service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:08.146962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a6df6700-495e-4ce1-b006-df069ca36f5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:08.308107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df6700-495e-4ce1-b006-df069ca36f5e service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:08.314338 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f41d5-bfbd-44d6-a0a4-7d9cf181deb3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:08.315047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f41d5-bfbd-44d6-a0a4-7d9cf181deb3 service swift] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:08.315366 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f41d5-bfbd-44d6-a0a4-7d9cf181deb3 service swift] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:08.316573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f41d5-bfbd-44d6-a0a4-7d9cf181deb3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:08.326426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f41d5-bfbd-44d6-a0a4-7d9cf181deb3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:08.328259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f41d5-bfbd-44d6-a0a4-7d9cf181deb3 service swift] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:08.338825 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1237/2481] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:25:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:25:18.048198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=40 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:18.336054 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=71 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:25.907564 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0e22d-1770-4ba5-aa48-76a875bf3819 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:25.908395 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0e22d-1770-4ba5-aa48-76a875bf3819 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:25.908605 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0e22d-1770-4ba5-aa48-76a875bf3819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:25.942371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-44a0e22d-1770-4ba5-aa48-76a875bf3819 None None] MFA Rules not processed for user `49f98078ac424b1284932839be652265`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:25.945545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-44a0e22d-1770-4ba5-aa48-76a875bf3819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:26.012822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1245/2482] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:25:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:27.969052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15e8fb-12a1-4fc3-a66c-72a7a45a8ee6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:27.969635 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15e8fb-12a1-4fc3-a66c-72a7a45a8ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:27.969789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15e8fb-12a1-4fc3-a66c-72a7a45a8ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:27.991118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4c15e8fb-12a1-4fc3-a66c-72a7a45a8ee6 None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:27.992350 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4c15e8fb-12a1-4fc3-a66c-72a7a45a8ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:28.001824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1238/2483] 10.222.0.100 () {58 vars in 1014 bytes} [Mon Aug 1 15:25:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:28.017032 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0e22d-1770-4ba5-aa48-76a875bf3819 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:28.031783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0e22d-1770-4ba5-aa48-76a875bf3819 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:28.038430 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af615b0-25b2-4a9a-afde-f40e297255ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:28.039360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af615b0-25b2-4a9a-afde-f40e297255ad service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:28.039865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af615b0-25b2-4a9a-afde-f40e297255ad service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:28.040153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af615b0-25b2-4a9a-afde-f40e297255ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:28.052307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af615b0-25b2-4a9a-afde-f40e297255ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:28.053904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af615b0-25b2-4a9a-afde-f40e297255ad service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:28.063648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1246/2484] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:25:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 15:25:32.260380 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15e8fb-12a1-4fc3-a66c-72a7a45a8ee6 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:32.262070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4c15e8fb-12a1-4fc3-a66c-72a7a45a8ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:32.279206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15e8fb-12a1-4fc3-a66c-72a7a45a8ee6 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:32.285096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc97a9f-88b3-4caa-aed5-011891ae6f0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:32.285832 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc97a9f-88b3-4caa-aed5-011891ae6f0a service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:32.286220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc97a9f-88b3-4caa-aed5-011891ae6f0a service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:32.286611 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc97a9f-88b3-4caa-aed5-011891ae6f0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:32.288695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0fc97a9f-88b3-4caa-aed5-011891ae6f0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:32.399029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc97a9f-88b3-4caa-aed5-011891ae6f0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:32.400767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc97a9f-88b3-4caa-aed5-011891ae6f0a service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:32.425087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1239/2485] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:25:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:25:38.059791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=31 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:42.421556 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=44 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:25:49.412905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af615b0-25b2-4a9a-afde-f40e297255ad service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:49.424068 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af615b0-25b2-4a9a-afde-f40e297255ad service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:49.427982 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62de154-d1dc-4278-a87b-a5cd60439582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:49.428713 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62de154-d1dc-4278-a87b-a5cd60439582 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:49.429073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62de154-d1dc-4278-a87b-a5cd60439582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:49.429423 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62de154-d1dc-4278-a87b-a5cd60439582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:49.431732 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d62de154-d1dc-4278-a87b-a5cd60439582 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:49.445367 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62de154-d1dc-4278-a87b-a5cd60439582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:49.447080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62de154-d1dc-4278-a87b-a5cd60439582 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:49.457783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1247/2486] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:25:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:50.833500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc97a9f-88b3-4caa-aed5-011891ae6f0a service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:50.852263 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc97a9f-88b3-4caa-aed5-011891ae6f0a service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:50.858144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7189d22-46ee-493f-a0da-60dc9642cf3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:50.858832 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7189d22-46ee-493f-a0da-60dc9642cf3a service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:50.859127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7189d22-46ee-493f-a0da-60dc9642cf3a service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:50.859411 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7189d22-46ee-493f-a0da-60dc9642cf3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:50.861240 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a7189d22-46ee-493f-a0da-60dc9642cf3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:50.874642 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7189d22-46ee-493f-a0da-60dc9642cf3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:50.876322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7189d22-46ee-493f-a0da-60dc9642cf3a service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:50.888464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1240/2487] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:25:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:51.866482 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62de154-d1dc-4278-a87b-a5cd60439582 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:51.881725 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62de154-d1dc-4278-a87b-a5cd60439582 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:51.887334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66b46e-ebfc-4237-91e6-90ecd1367bec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:51.887959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66b46e-ebfc-4237-91e6-90ecd1367bec service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:51.888241 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66b46e-ebfc-4237-91e6-90ecd1367bec service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:51.888518 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66b46e-ebfc-4237-91e6-90ecd1367bec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:51.890290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3b66b46e-ebfc-4237-91e6-90ecd1367bec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:51.903272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66b46e-ebfc-4237-91e6-90ecd1367bec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:51.904865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66b46e-ebfc-4237-91e6-90ecd1367bec service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:51.917088 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1248/2488] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:25:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:25:56.858788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778a1c3-8f19-45d2-bd75-ce9495589b0a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:56.859271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778a1c3-8f19-45d2-bd75-ce9495589b0a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:56.859416 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778a1c3-8f19-45d2-bd75-ce9495589b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:56.881021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7778a1c3-8f19-45d2-bd75-ce9495589b0a None None] MFA Rules not processed for user `d0723b88d7cd481cb962072f75d835c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:25:56.882469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7778a1c3-8f19-45d2-bd75-ce9495589b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:25:56.890561 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1241/2489] 10.222.0.100 () {58 vars in 972 bytes} [Mon Aug 1 15:25:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:25:56.902758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66b46e-ebfc-4237-91e6-90ecd1367bec service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:25:56.914196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66b46e-ebfc-4237-91e6-90ecd1367bec service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:25:56.919818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bc531-87b8-4b1e-b47e-8d8220be67a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a399a3cc7446c6a8dad94075851797', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:25:56.920434 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bc531-87b8-4b1e-b47e-8d8220be67a6 service swift] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:25:56.920781 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bc531-87b8-4b1e-b47e-8d8220be67a6 service swift] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:25:56.921060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bc531-87b8-4b1e-b47e-8d8220be67a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:25:56.929758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bc531-87b8-4b1e-b47e-8d8220be67a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:25:56.931371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386bc531-87b8-4b1e-b47e-8d8220be67a6 service swift] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:25:56.940100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1249/2490] 10.222.0.100 () {60 vars in 1389 bytes} [Mon Aug 1 15:25:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 15:26:02.561224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778a1c3-8f19-45d2-bd75-ce9495589b0a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:02.575001 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778a1c3-8f19-45d2-bd75-ce9495589b0a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:02.581082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a85db-6d10-44b7-9288-253bc5fd2cb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:02.581716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a85db-6d10-44b7-9288-253bc5fd2cb6 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:02.582003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a85db-6d10-44b7-9288-253bc5fd2cb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:02.582276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a85db-6d10-44b7-9288-253bc5fd2cb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:02.584132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7d5a85db-6d10-44b7-9288-253bc5fd2cb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:02.598355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a85db-6d10-44b7-9288-253bc5fd2cb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:02.600040 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a85db-6d10-44b7-9288-253bc5fd2cb6 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:02.611736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1242/2491] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:26:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:06.936837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=18 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:12.606845 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=15 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:23.735334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bc531-87b8-4b1e-b47e-8d8220be67a6 service swift] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:23.736904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-386bc531-87b8-4b1e-b47e-8d8220be67a6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:23.750169 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386bc531-87b8-4b1e-b47e-8d8220be67a6 service swift] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:23.754224 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444756e-f3e7-462b-bc93-c1fd05d1d5ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:23.754960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444756e-f3e7-462b-bc93-c1fd05d1d5ea service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:23.755331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444756e-f3e7-462b-bc93-c1fd05d1d5ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:23.755630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444756e-f3e7-462b-bc93-c1fd05d1d5ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:23.757616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8444756e-f3e7-462b-bc93-c1fd05d1d5ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:23.822413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444756e-f3e7-462b-bc93-c1fd05d1d5ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:23.823827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444756e-f3e7-462b-bc93-c1fd05d1d5ea service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:23.834870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1250/2492] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:27.148561 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a85db-6d10-44b7-9288-253bc5fd2cb6 service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:27.150091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7d5a85db-6d10-44b7-9288-253bc5fd2cb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:27.167379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a85db-6d10-44b7-9288-253bc5fd2cb6 service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:27.171951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970835c-6371-4250-8d4a-a9612be7f683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:27.172590 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970835c-6371-4250-8d4a-a9612be7f683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:27.172875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970835c-6371-4250-8d4a-a9612be7f683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:27.173176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970835c-6371-4250-8d4a-a9612be7f683 admin admin] PATH_INFO: `/v3/users/f33ea26c1f72445bb4cb50ea503600dc` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:27.174343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970835c-6371-4250-8d4a-a9612be7f683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f33ea26c1f72445bb4cb50ea503600dc)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:27.176351 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970835c-6371-4250-8d4a-a9612be7f683 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:27.225718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6970835c-6371-4250-8d4a-a9612be7f683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f33ea26c1f72445bb4cb50ea503600dc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:27.233669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6970835c-6371-4250-8d4a-a9612be7f683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f33ea26c1f72445bb4cb50ea503600dc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:27.234343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6970835c-6371-4250-8d4a-a9612be7f683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f33ea26c1f72445bb4cb50ea503600dc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:27.234974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6970835c-6371-4250-8d4a-a9612be7f683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f33ea26c1f72445bb4cb50ea503600dc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:27.247110 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6970835c-6371-4250-8d4a-a9612be7f683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f33ea26c1f72445bb4cb50ea503600dc'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:27.260243 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1243/2493] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:26:27 2022] DELETE /identity/v3/users/f33ea26c1f72445bb4cb50ea503600dc => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:27.439767 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444756e-f3e7-462b-bc93-c1fd05d1d5ea service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:27.453571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444756e-f3e7-462b-bc93-c1fd05d1d5ea service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:27.512375 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:27.513026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:27.513901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:27.514184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed admin admin] PATH_INFO: `/v3/projects/18f123f171554a58a2dea8fcc2afbefd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:27.515394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=18f123f171554a58a2dea8fcc2afbefd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:27.517311 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:27.571638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18f123f171554a58a2dea8fcc2afbefd', 'request_id': 'req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:27.579005 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18f123f171554a58a2dea8fcc2afbefd', 'request_id': 'req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:27.584656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18f123f171554a58a2dea8fcc2afbefd', 'request_id': 'req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:27.597171 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed admin admin] The token cache is being invalidate because project 18f123f171554a58a2dea8fcc2afbefd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:27.597463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:27.598188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b7de5dd-7146-4ce0-877e-28d51ceec1ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:27.599730 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1251/2494] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:26:27 2022] DELETE /identity/v3/projects/18f123f171554a58a2dea8fcc2afbefd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:27.605015 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da485b-0f86-4aea-99da-7db21bfde16a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:27.605305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da485b-0f86-4aea-99da-7db21bfde16a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:27.605587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da485b-0f86-4aea-99da-7db21bfde16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:27.626481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e6da485b-0f86-4aea-99da-7db21bfde16a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:27.627750 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e6da485b-0f86-4aea-99da-7db21bfde16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:27.688216 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1244/2495] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:27.697158 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283a0e-064e-48df-8f1b-049d3eb66f88 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:27.697446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283a0e-064e-48df-8f1b-049d3eb66f88 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:27.697730 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283a0e-064e-48df-8f1b-049d3eb66f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:27.719517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5c283a0e-064e-48df-8f1b-049d3eb66f88 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:27.720753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5c283a0e-064e-48df-8f1b-049d3eb66f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:27.728623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1252/2496] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:27.733397 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da485b-0f86-4aea-99da-7db21bfde16a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:27.745994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da485b-0f86-4aea-99da-7db21bfde16a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:27.751548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c75407-bb6c-4ed5-b068-e47b2c4215e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:27.752158 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c75407-bb6c-4ed5-b068-e47b2c4215e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:27.752437 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c75407-bb6c-4ed5-b068-e47b2c4215e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:27.752719 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c75407-bb6c-4ed5-b068-e47b2c4215e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:27.753409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c75407-bb6c-4ed5-b068-e47b2c4215e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:26:27.753775 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c75407-bb6c-4ed5-b068-e47b2c4215e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:27.755424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c75407-bb6c-4ed5-b068-e47b2c4215e1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:27.767229 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1245/2497] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:26:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:26:27.772262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c283a0e-064e-48df-8f1b-049d3eb66f88 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:27.781682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c283a0e-064e-48df-8f1b-049d3eb66f88 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:27.786437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d77b11-8f61-405b-94f5-a5271d816955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:27.787082 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d77b11-8f61-405b-94f5-a5271d816955 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:27.787353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d77b11-8f61-405b-94f5-a5271d816955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:27.787623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d77b11-8f61-405b-94f5-a5271d816955 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:27.788293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d77b11-8f61-405b-94f5-a5271d816955 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:27.790254 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d77b11-8f61-405b-94f5-a5271d816955 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:27.806224 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:27.821568 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1253/2498] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:26:27 2022] POST /identity/v3/projects => generated 356 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:27.826166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c75407-bb6c-4ed5-b068-e47b2c4215e1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:27.836479 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c75407-bb6c-4ed5-b068-e47b2c4215e1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:27.897100 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3b7be-9cf1-4a43-b9bb-18f7b1b9cb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:27.897724 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3b7be-9cf1-4a43-b9bb-18f7b1b9cb94 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:27.898013 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3b7be-9cf1-4a43-b9bb-18f7b1b9cb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:27.898290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3b7be-9cf1-4a43-b9bb-18f7b1b9cb94 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:27.898998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3b7be-9cf1-4a43-b9bb-18f7b1b9cb94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:27.900947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3b7be-9cf1-4a43-b9bb-18f7b1b9cb94 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:27.908947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:27.925594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1246/2499] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:26:27 2022] POST /identity/v3/users => generated 279 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:27.930248 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d77b11-8f61-405b-94f5-a5271d816955 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:27.942440 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d77b11-8f61-405b-94f5-a5271d816955 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:27.947496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1af2c6-b13c-445e-90e0-b579b22fca85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:27.948186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1af2c6-b13c-445e-90e0-b579b22fca85 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:27.948510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1af2c6-b13c-445e-90e0-b579b22fca85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:27.948831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1af2c6-b13c-445e-90e0-b579b22fca85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:27.949630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1af2c6-b13c-445e-90e0-b579b22fca85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:27.952268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1af2c6-b13c-445e-90e0-b579b22fca85 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:27.964276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1254/2500] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:26:27 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:27.969334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3b7be-9cf1-4a43-b9bb-18f7b1b9cb94 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:27.980258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3b7be-9cf1-4a43-b9bb-18f7b1b9cb94 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:27.985146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e67b62-26ac-41bf-9fc5-5776d6a052f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:27.985855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e67b62-26ac-41bf-9fc5-5776d6a052f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:27.986134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e67b62-26ac-41bf-9fc5-5776d6a052f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:27.986411 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e67b62-26ac-41bf-9fc5-5776d6a052f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:27.987133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e67b62-26ac-41bf-9fc5-5776d6a052f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:27.988733 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e67b62-26ac-41bf-9fc5-5776d6a052f2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:28.003454 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1247/2501] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:26:27 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:28.007957 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1af2c6-b13c-445e-90e0-b579b22fca85 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:28.019139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1af2c6-b13c-445e-90e0-b579b22fca85 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:28.024223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35fce7-4efb-403f-80de-93bac273dd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:28.025114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35fce7-4efb-403f-80de-93bac273dd87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:28.025494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35fce7-4efb-403f-80de-93bac273dd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:28.025876 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b35fce7-4efb-403f-80de-93bac273dd87 admin admin] PATH_INFO: `/v3/projects/c1ef9814afc34019bd06b5a77613fa43/users/a6e93583f77f49fa89b0b8140b1029fb/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:28.042995 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b35fce7-4efb-403f-80de-93bac273dd87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ef9814afc34019bd06b5a77613fa43, user_id=a6e93583f77f49fa89b0b8140b1029fb, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:28.046788 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b35fce7-4efb-403f-80de-93bac273dd87 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:28.062375 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1255/2502] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:26:28 2022] PUT /identity/v3/projects/c1ef9814afc34019bd06b5a77613fa43/users/a6e93583f77f49fa89b0b8140b1029fb/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:28.073348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b130-20fa-4e76-bd38-9d9fe90d1102 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:28.073638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b130-20fa-4e76-bd38-9d9fe90d1102 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:28.073919 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b130-20fa-4e76-bd38-9d9fe90d1102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:28.102267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-68d3b130-20fa-4e76-bd38-9d9fe90d1102 None None] MFA Rules not processed for user `a6e93583f77f49fa89b0b8140b1029fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:28.107490 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-68d3b130-20fa-4e76-bd38-9d9fe90d1102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:28.174542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1248/2503] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:28.179278 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35fce7-4efb-403f-80de-93bac273dd87 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:28.244894 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b35fce7-4efb-403f-80de-93bac273dd87 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:28.250541 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f7399-627d-42fc-842a-326afd221e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:28.251197 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f7399-627d-42fc-842a-326afd221e40 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:28.251481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f7399-627d-42fc-842a-326afd221e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:28.251757 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f7399-627d-42fc-842a-326afd221e40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:28.252443 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f7399-627d-42fc-842a-326afd221e40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:28.254368 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f7399-627d-42fc-842a-326afd221e40 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:28.284829 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1256/2504] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:26:28 2022] POST /identity/v3/projects => generated 358 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:28.288980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3b130-20fa-4e76-bd38-9d9fe90d1102 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:28.300956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3b130-20fa-4e76-bd38-9d9fe90d1102 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:28.359311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8918e-17e9-4708-bc50-fb8f05650726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:28.360103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8918e-17e9-4708-bc50-fb8f05650726 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:28.360393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8918e-17e9-4708-bc50-fb8f05650726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:28.360670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8918e-17e9-4708-bc50-fb8f05650726 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:28.361350 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8918e-17e9-4708-bc50-fb8f05650726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:28.363303 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8918e-17e9-4708-bc50-fb8f05650726 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:28.387329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1249/2505] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:26:28 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:28.392539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f7399-627d-42fc-842a-326afd221e40 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:28.404909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f7399-627d-42fc-842a-326afd221e40 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:28.409645 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40ffd2-ac59-4cde-ba8d-acb17e8e5d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:28.410259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40ffd2-ac59-4cde-ba8d-acb17e8e5d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:28.410543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40ffd2-ac59-4cde-ba8d-acb17e8e5d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:28.410865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40ffd2-ac59-4cde-ba8d-acb17e8e5d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:28.411550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40ffd2-ac59-4cde-ba8d-acb17e8e5d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:28.413158 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40ffd2-ac59-4cde-ba8d-acb17e8e5d12 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:28.424667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1257/2506] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:26:28 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:28.429417 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8918e-17e9-4708-bc50-fb8f05650726 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:28.439609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8918e-17e9-4708-bc50-fb8f05650726 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:28.444266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee42f1-c7c7-49bf-b7fa-1c8f20a2ba08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:28.444933 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee42f1-c7c7-49bf-b7fa-1c8f20a2ba08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:28.445215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee42f1-c7c7-49bf-b7fa-1c8f20a2ba08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:28.445496 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee42f1-c7c7-49bf-b7fa-1c8f20a2ba08 admin admin] PATH_INFO: `/v3/projects/48c850cd4bb844de9ebd5b7fe93586a5/users/7b7e5f4bda39460a8d1fe4b7f44ffb00/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:28.461160 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee42f1-c7c7-49bf-b7fa-1c8f20a2ba08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48c850cd4bb844de9ebd5b7fe93586a5, user_id=7b7e5f4bda39460a8d1fe4b7f44ffb00, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:28.463817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee42f1-c7c7-49bf-b7fa-1c8f20a2ba08 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:28.479165 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1250/2507] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:26:28 2022] PUT /identity/v3/projects/48c850cd4bb844de9ebd5b7fe93586a5/users/7b7e5f4bda39460a8d1fe4b7f44ffb00/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:28.483361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40ffd2-ac59-4cde-ba8d-acb17e8e5d12 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:28.493827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40ffd2-ac59-4cde-ba8d-acb17e8e5d12 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:28.551414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd697f66-a00c-48f1-8ddc-e0778450f703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:28.552040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd697f66-a00c-48f1-8ddc-e0778450f703 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:28.552325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd697f66-a00c-48f1-8ddc-e0778450f703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:28.552606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd697f66-a00c-48f1-8ddc-e0778450f703 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:28.553304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd697f66-a00c-48f1-8ddc-e0778450f703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:28.554978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd697f66-a00c-48f1-8ddc-e0778450f703 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:28.565227 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1258/2508] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:26:28 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:28.569784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee42f1-c7c7-49bf-b7fa-1c8f20a2ba08 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:28.581581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee42f1-c7c7-49bf-b7fa-1c8f20a2ba08 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:28.586342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f37ec-ea88-4874-bd36-af81a0c1eb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:28.587053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f37ec-ea88-4874-bd36-af81a0c1eb8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:28.587337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f37ec-ea88-4874-bd36-af81a0c1eb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:28.587621 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f37ec-ea88-4874-bd36-af81a0c1eb8f admin admin] PATH_INFO: `/v3/projects/48c850cd4bb844de9ebd5b7fe93586a5/users/7b7e5f4bda39460a8d1fe4b7f44ffb00/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:28.592816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f37ec-ea88-4874-bd36-af81a0c1eb8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=48c850cd4bb844de9ebd5b7fe93586a5, user_id=7b7e5f4bda39460a8d1fe4b7f44ffb00, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:28.595530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f37ec-ea88-4874-bd36-af81a0c1eb8f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:28.605980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1251/2509] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:26:28 2022] PUT /identity/v3/projects/48c850cd4bb844de9ebd5b7fe93586a5/users/7b7e5f4bda39460a8d1fe4b7f44ffb00/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:28.616874 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78704cd-e819-420b-887e-6e1c3e9a430e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:28.617196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78704cd-e819-420b-887e-6e1c3e9a430e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:28.617483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78704cd-e819-420b-887e-6e1c3e9a430e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:28.646479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a78704cd-e819-420b-887e-6e1c3e9a430e None None] MFA Rules not processed for user `7b7e5f4bda39460a8d1fe4b7f44ffb00`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:28.647852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a78704cd-e819-420b-887e-6e1c3e9a430e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:28.667508 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:28.727899 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1259/2510] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:26:28 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:28.739736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f37ec-ea88-4874-bd36-af81a0c1eb8f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:28.741455 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-377f37ec-ea88-4874-bd36-af81a0c1eb8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:28.765268 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:28.803572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f37ec-ea88-4874-bd36-af81a0c1eb8f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:28.807278 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296e4db-3580-43d2-b1af-7c6238c53b5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:28.807893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296e4db-3580-43d2-b1af-7c6238c53b5d service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:28.808177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296e4db-3580-43d2-b1af-7c6238c53b5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:28.808457 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296e4db-3580-43d2-b1af-7c6238c53b5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:28.818209 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296e4db-3580-43d2-b1af-7c6238c53b5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:28.819677 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296e4db-3580-43d2-b1af-7c6238c53b5d service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:28.829947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1252/2511] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:26:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:28.905573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78704cd-e819-420b-887e-6e1c3e9a430e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:28.915467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78704cd-e819-420b-887e-6e1c3e9a430e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:28.919112 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67afab4-64c6-4fa3-b934-1007c60a345d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:28.919722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67afab4-64c6-4fa3-b934-1007c60a345d service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:28.920002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67afab4-64c6-4fa3-b934-1007c60a345d service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:28.920276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67afab4-64c6-4fa3-b934-1007c60a345d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:28.980035 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67afab4-64c6-4fa3-b934-1007c60a345d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:28.982344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67afab4-64c6-4fa3-b934-1007c60a345d service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:28.991722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1260/2512] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:26:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:38.825054 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=17 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:38.987528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=48 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:45.022481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296e4db-3580-43d2-b1af-7c6238c53b5d service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:45.090950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296e4db-3580-43d2-b1af-7c6238c53b5d service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:45.096579 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f4931-a902-4ec9-8357-add44f5d6c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:45.097302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f4931-a902-4ec9-8357-add44f5d6c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:45.097625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f4931-a902-4ec9-8357-add44f5d6c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:45.097947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f4931-a902-4ec9-8357-add44f5d6c6a admin admin] PATH_INFO: `/v3/users/a6e93583f77f49fa89b0b8140b1029fb` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:45.099303 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f4931-a902-4ec9-8357-add44f5d6c6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6e93583f77f49fa89b0b8140b1029fb)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:45.101542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f4931-a902-4ec9-8357-add44f5d6c6a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:45.152830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3f5f4931-a902-4ec9-8357-add44f5d6c6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6e93583f77f49fa89b0b8140b1029fb'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.160316 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3f5f4931-a902-4ec9-8357-add44f5d6c6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6e93583f77f49fa89b0b8140b1029fb'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.160947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3f5f4931-a902-4ec9-8357-add44f5d6c6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6e93583f77f49fa89b0b8140b1029fb'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.161556 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3f5f4931-a902-4ec9-8357-add44f5d6c6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6e93583f77f49fa89b0b8140b1029fb'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.174152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3f5f4931-a902-4ec9-8357-add44f5d6c6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6e93583f77f49fa89b0b8140b1029fb'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.188672 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1253/2513] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:26:45 2022] DELETE /identity/v3/users/a6e93583f77f49fa89b0b8140b1029fb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:45.193437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a810630-031d-4123-ac2c-0b25052e9f5f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:45.193949 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a810630-031d-4123-ac2c-0b25052e9f5f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:45.194066 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a810630-031d-4123-ac2c-0b25052e9f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:45.213791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-2a810630-031d-4123-ac2c-0b25052e9f5f None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:45.215031 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2a810630-031d-4123-ac2c-0b25052e9f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:45.280494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1261/2514] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:45.292145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f4931-a902-4ec9-8357-add44f5d6c6a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:45.294940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3f5f4931-a902-4ec9-8357-add44f5d6c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:45.353640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f4931-a902-4ec9-8357-add44f5d6c6a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:45.357267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecb8fa-3d5d-44f6-9420-284db1e49b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:45.357877 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecb8fa-3d5d-44f6-9420-284db1e49b73 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:45.358156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecb8fa-3d5d-44f6-9420-284db1e49b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:45.358429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecb8fa-3d5d-44f6-9420-284db1e49b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:45.367298 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ecb8fa-3d5d-44f6-9420-284db1e49b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:45.368731 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ecb8fa-3d5d-44f6-9420-284db1e49b73 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:45.379588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1254/2515] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:26:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:45.667007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a810630-031d-4123-ac2c-0b25052e9f5f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:45.668449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2a810630-031d-4123-ac2c-0b25052e9f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:45.681774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a810630-031d-4123-ac2c-0b25052e9f5f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:45.686509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:45.687253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:45.687508 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:45.687751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] PATH_INFO: `/v3/projects/c1ef9814afc34019bd06b5a77613fa43` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:45.689749 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ef9814afc34019bd06b5a77613fa43)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:45.691781 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:45.739409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ef9814afc34019bd06b5a77613fa43', 'request_id': 'req-99ca279f-4d8c-448b-b43f-0dfd139635bc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.745693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ef9814afc34019bd06b5a77613fa43', 'request_id': 'req-99ca279f-4d8c-448b-b43f-0dfd139635bc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.752086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ef9814afc34019bd06b5a77613fa43', 'request_id': 'req-99ca279f-4d8c-448b-b43f-0dfd139635bc'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.764479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] The token cache is being invalidate because project c1ef9814afc34019bd06b5a77613fa43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:45.764776 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.765396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.766966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1262/2516] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:26:45 2022] DELETE /identity/v3/projects/c1ef9814afc34019bd06b5a77613fa43 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:45.771147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecb8fa-3d5d-44f6-9420-284db1e49b73 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:45.780950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecb8fa-3d5d-44f6-9420-284db1e49b73 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:45.800173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:45.841187 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb6149-da7d-445b-ae6d-b697c3655c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:45.841820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb6149-da7d-445b-ae6d-b697c3655c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:45.842098 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb6149-da7d-445b-ae6d-b697c3655c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:45.842372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb6149-da7d-445b-ae6d-b697c3655c07 admin admin] PATH_INFO: `/v3/users/7b7e5f4bda39460a8d1fe4b7f44ffb00` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:45.843660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb6149-da7d-445b-ae6d-b697c3655c07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b7e5f4bda39460a8d1fe4b7f44ffb00)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:45.845716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb6149-da7d-445b-ae6d-b697c3655c07 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:45.898349 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-fceb6149-da7d-445b-ae6d-b697c3655c07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b7e5f4bda39460a8d1fe4b7f44ffb00'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.907983 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-fceb6149-da7d-445b-ae6d-b697c3655c07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b7e5f4bda39460a8d1fe4b7f44ffb00'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.908851 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-fceb6149-da7d-445b-ae6d-b697c3655c07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b7e5f4bda39460a8d1fe4b7f44ffb00'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.909698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-fceb6149-da7d-445b-ae6d-b697c3655c07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b7e5f4bda39460a8d1fe4b7f44ffb00'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.926892 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-fceb6149-da7d-445b-ae6d-b697c3655c07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b7e5f4bda39460a8d1fe4b7f44ffb00'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:45.945207 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1255/2517] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:26:45 2022] DELETE /identity/v3/users/7b7e5f4bda39460a8d1fe4b7f44ffb00 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:46.228897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:46.231119 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:46.245398 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca279f-4d8c-448b-b43f-0dfd139635bc admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:46.317801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:46.318651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:46.319085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:46.319460 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2 admin admin] PATH_INFO: `/v3/projects/48c850cd4bb844de9ebd5b7fe93586a5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:46.321080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48c850cd4bb844de9ebd5b7fe93586a5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:46.323682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:46.358285 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:46.375764 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48c850cd4bb844de9ebd5b7fe93586a5', 'request_id': 'req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:46.382916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48c850cd4bb844de9ebd5b7fe93586a5', 'request_id': 'req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:46.388622 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48c850cd4bb844de9ebd5b7fe93586a5', 'request_id': 'req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:46.401048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2 admin admin] The token cache is being invalidate because project 48c850cd4bb844de9ebd5b7fe93586a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:26:46.401337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:46.401964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-60250a19-a0e8-4cb6-b3fc-3a2a6895e8e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:26:46.403510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1263/2518] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:26:46 2022] DELETE /identity/v3/projects/48c850cd4bb844de9ebd5b7fe93586a5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:46.409082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd47aa-b366-40f9-8e02-4e793771981d None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:46.409583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd47aa-b366-40f9-8e02-4e793771981d None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:46.409691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd47aa-b366-40f9-8e02-4e793771981d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:46.430588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-6dbd47aa-b366-40f9-8e02-4e793771981d None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:46.431751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6dbd47aa-b366-40f9-8e02-4e793771981d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:46.493476 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1256/2519] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:26:46.502050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c97aa-e998-4c88-a0aa-3d405278bf2e None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:46.502341 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c97aa-e998-4c88-a0aa-3d405278bf2e None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:46.502622 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c97aa-e998-4c88-a0aa-3d405278bf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:46.522207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-577c97aa-e998-4c88-a0aa-3d405278bf2e None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:46.523487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-577c97aa-e998-4c88-a0aa-3d405278bf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:46.534678 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1264/2520] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:46.536240 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd47aa-b366-40f9-8e02-4e793771981d None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:46.546700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd47aa-b366-40f9-8e02-4e793771981d None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:46.552005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23c17c-699a-48ca-bfd7-88dc42448dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:46.552622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23c17c-699a-48ca-bfd7-88dc42448dfa admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:46.552899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23c17c-699a-48ca-bfd7-88dc42448dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:46.553168 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23c17c-699a-48ca-bfd7-88dc42448dfa admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:46.553863 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23c17c-699a-48ca-bfd7-88dc42448dfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:26:46.554167 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23c17c-699a-48ca-bfd7-88dc42448dfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:46.555809 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23c17c-699a-48ca-bfd7-88dc42448dfa admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:46.568307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1257/2521] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:26:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:26:46.572592 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c97aa-e998-4c88-a0aa-3d405278bf2e None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:46.582531 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c97aa-e998-4c88-a0aa-3d405278bf2e None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:46.591714 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae671fe-9257-4d95-aa18-2b830079476a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:46.592363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae671fe-9257-4d95-aa18-2b830079476a admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:46.592648 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae671fe-9257-4d95-aa18-2b830079476a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:46.592931 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae671fe-9257-4d95-aa18-2b830079476a admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:46.593629 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae671fe-9257-4d95-aa18-2b830079476a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:46.595639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae671fe-9257-4d95-aa18-2b830079476a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:46.626468 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1265/2522] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:26:46 2022] POST /identity/v3/projects => generated 376 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:46.630936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23c17c-699a-48ca-bfd7-88dc42448dfa admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:46.640871 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23c17c-699a-48ca-bfd7-88dc42448dfa admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:46.667217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:46.704282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c826bd-6665-4747-9d91-9d0be997fe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:46.704815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c826bd-6665-4747-9d91-9d0be997fe73 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:46.705101 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c826bd-6665-4747-9d91-9d0be997fe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:46.705381 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c826bd-6665-4747-9d91-9d0be997fe73 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:46.706048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c826bd-6665-4747-9d91-9d0be997fe73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:46.708132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c826bd-6665-4747-9d91-9d0be997fe73 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:46.730708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1258/2523] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:26:46 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:46.735779 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae671fe-9257-4d95-aa18-2b830079476a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:46.746028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae671fe-9257-4d95-aa18-2b830079476a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:46.751174 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501b94a-d29d-447e-bf3e-ffd05239e85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:46.751779 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501b94a-d29d-447e-bf3e-ffd05239e85e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:46.752060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501b94a-d29d-447e-bf3e-ffd05239e85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:46.752334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501b94a-d29d-447e-bf3e-ffd05239e85e admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:46.753021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501b94a-d29d-447e-bf3e-ffd05239e85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:46.754610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501b94a-d29d-447e-bf3e-ffd05239e85e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:46.764328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1266/2524] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:26:46 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:46.769064 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c826bd-6665-4747-9d91-9d0be997fe73 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:46.779323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c826bd-6665-4747-9d91-9d0be997fe73 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:46.784061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1065fb2-ce6b-454a-8d9d-26adec2cc034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:46.784672 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1065fb2-ce6b-454a-8d9d-26adec2cc034 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:46.784958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1065fb2-ce6b-454a-8d9d-26adec2cc034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:46.785236 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1065fb2-ce6b-454a-8d9d-26adec2cc034 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:46.785933 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1065fb2-ce6b-454a-8d9d-26adec2cc034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:46.787631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1065fb2-ce6b-454a-8d9d-26adec2cc034 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:46.798896 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1259/2525] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:26:46 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:46.803290 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501b94a-d29d-447e-bf3e-ffd05239e85e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:46.814410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501b94a-d29d-447e-bf3e-ffd05239e85e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:46.819098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f30f60-9413-436f-8448-ce55ffd13221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:46.819761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f30f60-9413-436f-8448-ce55ffd13221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:46.820047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f30f60-9413-436f-8448-ce55ffd13221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:46.820327 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f30f60-9413-436f-8448-ce55ffd13221 admin admin] PATH_INFO: `/v3/projects/d74b6081bcfc4ff88d6ad92e1b370127/users/d14915b737a2485fbe5d6ff69d2e3797/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:46.835709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f30f60-9413-436f-8448-ce55ffd13221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d74b6081bcfc4ff88d6ad92e1b370127, user_id=d14915b737a2485fbe5d6ff69d2e3797, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:46.838397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f30f60-9413-436f-8448-ce55ffd13221 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:46.853072 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1267/2526] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:26:46 2022] PUT /identity/v3/projects/d74b6081bcfc4ff88d6ad92e1b370127/users/d14915b737a2485fbe5d6ff69d2e3797/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:26:46.864398 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9516dc-8b94-40b1-84ba-c8fcb62b33ab None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:46.864823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9516dc-8b94-40b1-84ba-c8fcb62b33ab None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:46.865200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9516dc-8b94-40b1-84ba-c8fcb62b33ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:46.898003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ed9516dc-8b94-40b1-84ba-c8fcb62b33ab None None] MFA Rules not processed for user `d14915b737a2485fbe5d6ff69d2e3797`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:46.899288 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ed9516dc-8b94-40b1-84ba-c8fcb62b33ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:46.967715 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1260/2527] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:26:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:46.972142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f30f60-9413-436f-8448-ce55ffd13221 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:47.036990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f30f60-9413-436f-8448-ce55ffd13221 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:47.041741 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45f27a-e184-401d-8238-4c61493a473f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:47.042362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45f27a-e184-401d-8238-4c61493a473f admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:47.042647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45f27a-e184-401d-8238-4c61493a473f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:47.042956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45f27a-e184-401d-8238-4c61493a473f admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:47.043625 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45f27a-e184-401d-8238-4c61493a473f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:47.045568 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45f27a-e184-401d-8238-4c61493a473f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:47.080219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1268/2528] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:26:46 2022] POST /identity/v3/projects => generated 378 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:26:47.084336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9516dc-8b94-40b1-84ba-c8fcb62b33ab None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:47.095332 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9516dc-8b94-40b1-84ba-c8fcb62b33ab None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:47.152804 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715fd070-4b82-4938-ad06-88d7bf8cccb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:47.153432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fd070-4b82-4938-ad06-88d7bf8cccb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:47.153723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fd070-4b82-4938-ad06-88d7bf8cccb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:47.154007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fd070-4b82-4938-ad06-88d7bf8cccb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:47.154723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715fd070-4b82-4938-ad06-88d7bf8cccb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:47.156662 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715fd070-4b82-4938-ad06-88d7bf8cccb4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:47.179532 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1261/2529] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:26:47 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:26:47.183641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45f27a-e184-401d-8238-4c61493a473f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:47.194993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45f27a-e184-401d-8238-4c61493a473f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:47.199708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd19bfe-588e-40a4-b59f-0f3ae8c5ee1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:47.200336 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd19bfe-588e-40a4-b59f-0f3ae8c5ee1a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:47.200882 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd19bfe-588e-40a4-b59f-0f3ae8c5ee1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:47.201161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd19bfe-588e-40a4-b59f-0f3ae8c5ee1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:47.201897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd19bfe-588e-40a4-b59f-0f3ae8c5ee1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:47.203552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd19bfe-588e-40a4-b59f-0f3ae8c5ee1a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:47.214640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1269/2530] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:26:47 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:47.219238 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715fd070-4b82-4938-ad06-88d7bf8cccb4 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:47.229216 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715fd070-4b82-4938-ad06-88d7bf8cccb4 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:47.233894 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf648488-22db-45dc-96c9-0b2843b02c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:47.234555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf648488-22db-45dc-96c9-0b2843b02c93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:47.235407 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf648488-22db-45dc-96c9-0b2843b02c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:47.235692 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf648488-22db-45dc-96c9-0b2843b02c93 admin admin] PATH_INFO: `/v3/projects/5c5c5763a3ac476a90ed1b1b9161e358/users/78eb2daffc06401083b02cf9c3ae1a6b/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:47.250305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf648488-22db-45dc-96c9-0b2843b02c93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c5c5763a3ac476a90ed1b1b9161e358, user_id=78eb2daffc06401083b02cf9c3ae1a6b, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:47.253558 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf648488-22db-45dc-96c9-0b2843b02c93 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:47.267383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1262/2531] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:26:47 2022] PUT /identity/v3/projects/5c5c5763a3ac476a90ed1b1b9161e358/users/78eb2daffc06401083b02cf9c3ae1a6b/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:47.272168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd19bfe-588e-40a4-b59f-0f3ae8c5ee1a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:47.283806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd19bfe-588e-40a4-b59f-0f3ae8c5ee1a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:47.304443 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:47.344627 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b91e43-b1d8-452f-9e62-ebae61cb5e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:47.345275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b91e43-b1d8-452f-9e62-ebae61cb5e9a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:47.345650 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b91e43-b1d8-452f-9e62-ebae61cb5e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:47.345933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b91e43-b1d8-452f-9e62-ebae61cb5e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:47.346638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b91e43-b1d8-452f-9e62-ebae61cb5e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:47.348279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b91e43-b1d8-452f-9e62-ebae61cb5e9a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:47.358267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1270/2532] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:26:47 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:26:47.363055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf648488-22db-45dc-96c9-0b2843b02c93 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:47.368619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:26:47.375308 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf648488-22db-45dc-96c9-0b2843b02c93 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:47.382076 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa52a3e-f773-4c0d-a300-f68fc24e83b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:47.382786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa52a3e-f773-4c0d-a300-f68fc24e83b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:47.383075 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa52a3e-f773-4c0d-a300-f68fc24e83b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:47.383360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa52a3e-f773-4c0d-a300-f68fc24e83b1 admin admin] PATH_INFO: `/v3/projects/5c5c5763a3ac476a90ed1b1b9161e358/users/78eb2daffc06401083b02cf9c3ae1a6b/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:47.385388 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa52a3e-f773-4c0d-a300-f68fc24e83b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c5c5763a3ac476a90ed1b1b9161e358, user_id=78eb2daffc06401083b02cf9c3ae1a6b, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:47.388128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa52a3e-f773-4c0d-a300-f68fc24e83b1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:47.398993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1263/2533] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:26:47 2022] PUT /identity/v3/projects/5c5c5763a3ac476a90ed1b1b9161e358/users/78eb2daffc06401083b02cf9c3ae1a6b/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:26:47.410762 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083d016-6eb5-431e-b33f-4cca10d8038b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:47.411103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083d016-6eb5-431e-b33f-4cca10d8038b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:47.411424 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083d016-6eb5-431e-b33f-4cca10d8038b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:47.439531 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c083d016-6eb5-431e-b33f-4cca10d8038b None None] MFA Rules not processed for user `78eb2daffc06401083b02cf9c3ae1a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:26:47.440785 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c083d016-6eb5-431e-b33f-4cca10d8038b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:47.511015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1271/2534] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:26:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:26:47.521780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa52a3e-f773-4c0d-a300-f68fc24e83b1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:47.523557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4aa52a3e-f773-4c0d-a300-f68fc24e83b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:26:47.583274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa52a3e-f773-4c0d-a300-f68fc24e83b1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:47.586890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead43101-1c1d-4fe2-b37e-9d9bc34b2b52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:47.587508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead43101-1c1d-4fe2-b37e-9d9bc34b2b52 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:47.587796 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead43101-1c1d-4fe2-b37e-9d9bc34b2b52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:47.588073 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead43101-1c1d-4fe2-b37e-9d9bc34b2b52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:47.645538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead43101-1c1d-4fe2-b37e-9d9bc34b2b52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:47.647040 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead43101-1c1d-4fe2-b37e-9d9bc34b2b52 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:47.656397 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1264/2535] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:26:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:56.441944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083d016-6eb5-431e-b33f-4cca10d8038b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:26:56.452135 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083d016-6eb5-431e-b33f-4cca10d8038b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:26:56.455716 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a231e9-3c59-4c41-aad6-7be1661c05dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:26:56.456323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a231e9-3c59-4c41-aad6-7be1661c05dd service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:26:56.456605 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a231e9-3c59-4c41-aad6-7be1661c05dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:26:56.456872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a231e9-3c59-4c41-aad6-7be1661c05dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:26:56.466412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a231e9-3c59-4c41-aad6-7be1661c05dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:26:56.467868 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a231e9-3c59-4c41-aad6-7be1661c05dd service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:26:56.478256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1272/2536] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:26:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:26:57.652176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:06.473880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=58 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:11.372122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead43101-1c1d-4fe2-b37e-9d9bc34b2b52 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:11.373549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ead43101-1c1d-4fe2-b37e-9d9bc34b2b52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:11.430402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead43101-1c1d-4fe2-b37e-9d9bc34b2b52 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:11.434144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae4fe08-76e5-43e1-9a9d-5c63069a57d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:11.434782 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4fe08-76e5-43e1-9a9d-5c63069a57d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:11.435070 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4fe08-76e5-43e1-9a9d-5c63069a57d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:11.435353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae4fe08-76e5-43e1-9a9d-5c63069a57d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:11.437206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4ae4fe08-76e5-43e1-9a9d-5c63069a57d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:11.517473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4fe08-76e5-43e1-9a9d-5c63069a57d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:11.518943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae4fe08-76e5-43e1-9a9d-5c63069a57d6 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:11.537817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1265/2537] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:21.535082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=61 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:26.855578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a231e9-3c59-4c41-aad6-7be1661c05dd service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:26.923051 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a231e9-3c59-4c41-aad6-7be1661c05dd service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:26.928753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85610b43-e3af-46a6-9e38-100fbad93a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:26.929766 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85610b43-e3af-46a6-9e38-100fbad93a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:26.929766 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85610b43-e3af-46a6-9e38-100fbad93a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:26.929959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85610b43-e3af-46a6-9e38-100fbad93a18 admin admin] PATH_INFO: `/v3/users/d14915b737a2485fbe5d6ff69d2e3797` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:26.931208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85610b43-e3af-46a6-9e38-100fbad93a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d14915b737a2485fbe5d6ff69d2e3797)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:26.933134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85610b43-e3af-46a6-9e38-100fbad93a18 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:26.997987 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-85610b43-e3af-46a6-9e38-100fbad93a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd14915b737a2485fbe5d6ff69d2e3797'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:26.999086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-85610b43-e3af-46a6-9e38-100fbad93a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd14915b737a2485fbe5d6ff69d2e3797'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:27.008870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-85610b43-e3af-46a6-9e38-100fbad93a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd14915b737a2485fbe5d6ff69d2e3797'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:27.009654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-85610b43-e3af-46a6-9e38-100fbad93a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd14915b737a2485fbe5d6ff69d2e3797'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:27.036688 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-85610b43-e3af-46a6-9e38-100fbad93a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd14915b737a2485fbe5d6ff69d2e3797'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:27.055831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1273/2538] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:27:26 2022] DELETE /identity/v3/users/d14915b737a2485fbe5d6ff69d2e3797 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:27.060325 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7915e8-151a-4b9d-a944-176b5c51f8f6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:27.060803 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7915e8-151a-4b9d-a944-176b5c51f8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:27.060944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7915e8-151a-4b9d-a944-176b5c51f8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:27.091944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-8c7915e8-151a-4b9d-a944-176b5c51f8f6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:27.093330 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8c7915e8-151a-4b9d-a944-176b5c51f8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:27.187504 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1266/2539] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:27.194570 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85610b43-e3af-46a6-9e38-100fbad93a18 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:27.196276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-85610b43-e3af-46a6-9e38-100fbad93a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:27.277557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85610b43-e3af-46a6-9e38-100fbad93a18 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:27.282156 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f53ad9-2113-41dd-a76d-89bccd924ee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:27.283355 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f53ad9-2113-41dd-a76d-89bccd924ee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:27.283639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f53ad9-2113-41dd-a76d-89bccd924ee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:27.283912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f53ad9-2113-41dd-a76d-89bccd924ee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:27.294031 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f53ad9-2113-41dd-a76d-89bccd924ee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:27.295670 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f53ad9-2113-41dd-a76d-89bccd924ee5 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:27.307073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1274/2540] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:27.643824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7915e8-151a-4b9d-a944-176b5c51f8f6 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:27.645285 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8c7915e8-151a-4b9d-a944-176b5c51f8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:27.662709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7915e8-151a-4b9d-a944-176b5c51f8f6 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:27.675141 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:27.675898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:27.676189 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:27.676468 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] PATH_INFO: `/v3/projects/d74b6081bcfc4ff88d6ad92e1b370127` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:27.678138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d74b6081bcfc4ff88d6ad92e1b370127)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:27.680271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:27.741221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd74b6081bcfc4ff88d6ad92e1b370127', 'request_id': 'req-12f9c876-4a3a-4676-a282-29913686f73b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:27.748530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd74b6081bcfc4ff88d6ad92e1b370127', 'request_id': 'req-12f9c876-4a3a-4676-a282-29913686f73b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:27.758127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd74b6081bcfc4ff88d6ad92e1b370127', 'request_id': 'req-12f9c876-4a3a-4676-a282-29913686f73b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:27.771321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] The token cache is being invalidate because project d74b6081bcfc4ff88d6ad92e1b370127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:27.771650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:27.772290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:27.774633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1267/2541] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:27:27 2022] DELETE /identity/v3/projects/d74b6081bcfc4ff88d6ad92e1b370127 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:27:27.778795 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f53ad9-2113-41dd-a76d-89bccd924ee5 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:27.789396 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f53ad9-2113-41dd-a76d-89bccd924ee5 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:27.808738 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:27.858572 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c9a76-9650-489e-b74a-5fe7568da304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:27.859409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c9a76-9650-489e-b74a-5fe7568da304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:27.859588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c9a76-9650-489e-b74a-5fe7568da304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:27.859868 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c9a76-9650-489e-b74a-5fe7568da304 admin admin] PATH_INFO: `/v3/users/78eb2daffc06401083b02cf9c3ae1a6b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:27.861100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c9a76-9650-489e-b74a-5fe7568da304 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78eb2daffc06401083b02cf9c3ae1a6b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:27.863182 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c9a76-9650-489e-b74a-5fe7568da304 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:27.923403 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a0c9a76-9650-489e-b74a-5fe7568da304 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78eb2daffc06401083b02cf9c3ae1a6b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:27.924120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a0c9a76-9650-489e-b74a-5fe7568da304 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78eb2daffc06401083b02cf9c3ae1a6b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:27.931773 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a0c9a76-9650-489e-b74a-5fe7568da304 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78eb2daffc06401083b02cf9c3ae1a6b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:27.932510 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a0c9a76-9650-489e-b74a-5fe7568da304 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78eb2daffc06401083b02cf9c3ae1a6b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:27.946083 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-1a0c9a76-9650-489e-b74a-5fe7568da304 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78eb2daffc06401083b02cf9c3ae1a6b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:27.963787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1275/2542] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:27:27 2022] DELETE /identity/v3/users/78eb2daffc06401083b02cf9c3ae1a6b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:28.518418 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:28.519925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:28.534215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9c876-4a3a-4676-a282-29913686f73b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:28.604962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788010e8-df6f-405a-b3d4-8282dafef9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:28.605599 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788010e8-df6f-405a-b3d4-8282dafef9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:28.605942 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788010e8-df6f-405a-b3d4-8282dafef9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:28.606137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788010e8-df6f-405a-b3d4-8282dafef9e8 admin admin] PATH_INFO: `/v3/projects/5c5c5763a3ac476a90ed1b1b9161e358` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:28.607318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788010e8-df6f-405a-b3d4-8282dafef9e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c5c5763a3ac476a90ed1b1b9161e358)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:28.610251 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788010e8-df6f-405a-b3d4-8282dafef9e8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:28.649362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:28.675584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-788010e8-df6f-405a-b3d4-8282dafef9e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c5c5763a3ac476a90ed1b1b9161e358', 'request_id': 'req-788010e8-df6f-405a-b3d4-8282dafef9e8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:28.683572 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-788010e8-df6f-405a-b3d4-8282dafef9e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c5c5763a3ac476a90ed1b1b9161e358', 'request_id': 'req-788010e8-df6f-405a-b3d4-8282dafef9e8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:28.692761 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-788010e8-df6f-405a-b3d4-8282dafef9e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c5c5763a3ac476a90ed1b1b9161e358', 'request_id': 'req-788010e8-df6f-405a-b3d4-8282dafef9e8'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:28.706517 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-788010e8-df6f-405a-b3d4-8282dafef9e8 admin admin] The token cache is being invalidate because project 5c5c5763a3ac476a90ed1b1b9161e358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:28.706842 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-788010e8-df6f-405a-b3d4-8282dafef9e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:28.707499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-788010e8-df6f-405a-b3d4-8282dafef9e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:28.709022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1268/2543] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:27:28 2022] DELETE /identity/v3/projects/5c5c5763a3ac476a90ed1b1b9161e358 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:28.714017 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89649478-8744-4b11-aac8-ff63aa761c22 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:28.714388 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89649478-8744-4b11-aac8-ff63aa761c22 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:28.714687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89649478-8744-4b11-aac8-ff63aa761c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:28.736676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-89649478-8744-4b11-aac8-ff63aa761c22 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:28.738098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-89649478-8744-4b11-aac8-ff63aa761c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:28.806711 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1276/2544] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:27:28.815133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1388a4-628e-436d-a46f-06a8dcd62e3b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:28.815413 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1388a4-628e-436d-a46f-06a8dcd62e3b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:28.815682 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1388a4-628e-436d-a46f-06a8dcd62e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:28.845241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-fc1388a4-628e-436d-a46f-06a8dcd62e3b None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:28.846583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fc1388a4-628e-436d-a46f-06a8dcd62e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:28.855168 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1269/2545] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:28.859673 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89649478-8744-4b11-aac8-ff63aa761c22 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:28.870452 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89649478-8744-4b11-aac8-ff63aa761c22 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:28.875114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95a1e6-a11f-4845-a9c2-e6ee82348a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:28.875760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95a1e6-a11f-4845-a9c2-e6ee82348a8c admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:28.876046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95a1e6-a11f-4845-a9c2-e6ee82348a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:28.876858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95a1e6-a11f-4845-a9c2-e6ee82348a8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:28.877552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95a1e6-a11f-4845-a9c2-e6ee82348a8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:27:28.877862 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95a1e6-a11f-4845-a9c2-e6ee82348a8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:28.879499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95a1e6-a11f-4845-a9c2-e6ee82348a8c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:28.891708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1277/2546] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:27:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:27:28.896633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1388a4-628e-436d-a46f-06a8dcd62e3b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:28.906350 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1388a4-628e-436d-a46f-06a8dcd62e3b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:28.911321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf91a2-c1f5-4fc9-99a8-1e415c96003f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:28.911922 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf91a2-c1f5-4fc9-99a8-1e415c96003f admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:28.912201 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf91a2-c1f5-4fc9-99a8-1e415c96003f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:28.912474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf91a2-c1f5-4fc9-99a8-1e415c96003f admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:28.913133 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf91a2-c1f5-4fc9-99a8-1e415c96003f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:28.915072 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf91a2-c1f5-4fc9-99a8-1e415c96003f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:28.946314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1270/2547] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:27:28 2022] POST /identity/v3/projects => generated 386 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:27:28.947443 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95a1e6-a11f-4845-a9c2-e6ee82348a8c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:28.958940 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95a1e6-a11f-4845-a9c2-e6ee82348a8c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:28.986162 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:29.018197 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf914c-2070-49d7-945b-ddeda67bc544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:29.018784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf914c-2070-49d7-945b-ddeda67bc544 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:29.019072 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf914c-2070-49d7-945b-ddeda67bc544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:29.019352 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf914c-2070-49d7-945b-ddeda67bc544 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:29.020033 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf914c-2070-49d7-945b-ddeda67bc544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:29.021955 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf914c-2070-49d7-945b-ddeda67bc544 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:29.042473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1278/2548] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:27:28 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:27:29.046495 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf91a2-c1f5-4fc9-99a8-1e415c96003f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:29.057326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf91a2-c1f5-4fc9-99a8-1e415c96003f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:29.062116 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356c4a7-3d23-47c8-a560-156a4251dec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:29.062759 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356c4a7-3d23-47c8-a560-156a4251dec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:29.063049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356c4a7-3d23-47c8-a560-156a4251dec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:29.063330 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356c4a7-3d23-47c8-a560-156a4251dec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:29.064589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356c4a7-3d23-47c8-a560-156a4251dec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:29.066192 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356c4a7-3d23-47c8-a560-156a4251dec4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:29.076544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1271/2549] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:29 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:29.080552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf914c-2070-49d7-945b-ddeda67bc544 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:29.090345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf914c-2070-49d7-945b-ddeda67bc544 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:29.095437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d09313-2902-4b3b-9c28-def8ddfc40ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:29.096094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d09313-2902-4b3b-9c28-def8ddfc40ad admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:29.096382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d09313-2902-4b3b-9c28-def8ddfc40ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:29.096665 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d09313-2902-4b3b-9c28-def8ddfc40ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:29.097374 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d09313-2902-4b3b-9c28-def8ddfc40ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:29.099104 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d09313-2902-4b3b-9c28-def8ddfc40ad admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:29.111294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1279/2550] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:29 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:29.115648 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356c4a7-3d23-47c8-a560-156a4251dec4 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:29.137229 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356c4a7-3d23-47c8-a560-156a4251dec4 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:29.142161 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e5826-9e59-4abd-91e9-39cfb9a726c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:29.143331 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e5826-9e59-4abd-91e9-39cfb9a726c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:29.143838 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e5826-9e59-4abd-91e9-39cfb9a726c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:29.144305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e5826-9e59-4abd-91e9-39cfb9a726c0 admin admin] PATH_INFO: `/v3/projects/e243242e5bfb46b48f2808db58ed8cbd/users/88fe6eb7ca7b48eea0a3555c6b36c31f/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:29.161824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e5826-9e59-4abd-91e9-39cfb9a726c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e243242e5bfb46b48f2808db58ed8cbd, user_id=88fe6eb7ca7b48eea0a3555c6b36c31f, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:29.164949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e5826-9e59-4abd-91e9-39cfb9a726c0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:29.189605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1272/2551] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:27:29 2022] PUT /identity/v3/projects/e243242e5bfb46b48f2808db58ed8cbd/users/88fe6eb7ca7b48eea0a3555c6b36c31f/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:27:29.194467 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01b627-ff23-4d2a-8c07-f4dbed1d9c89 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:29.194828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01b627-ff23-4d2a-8c07-f4dbed1d9c89 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:29.195107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01b627-ff23-4d2a-8c07-f4dbed1d9c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:29.227117 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6c01b627-ff23-4d2a-8c07-f4dbed1d9c89 None None] MFA Rules not processed for user `88fe6eb7ca7b48eea0a3555c6b36c31f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:29.228351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6c01b627-ff23-4d2a-8c07-f4dbed1d9c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:29.303433 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1280/2552] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:29.307878 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e5826-9e59-4abd-91e9-39cfb9a726c0 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:29.372683 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e5826-9e59-4abd-91e9-39cfb9a726c0 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:29.377336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06394dc7-0fff-431a-a4a7-f92c9fb2edbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:29.377965 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06394dc7-0fff-431a-a4a7-f92c9fb2edbf admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:29.378252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06394dc7-0fff-431a-a4a7-f92c9fb2edbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:29.378546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06394dc7-0fff-431a-a4a7-f92c9fb2edbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:29.379280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06394dc7-0fff-431a-a4a7-f92c9fb2edbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:29.381206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06394dc7-0fff-431a-a4a7-f92c9fb2edbf admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:29.409656 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1273/2553] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:27:29 2022] POST /identity/v3/projects => generated 386 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:27:29.413664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01b627-ff23-4d2a-8c07-f4dbed1d9c89 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:29.425561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01b627-ff23-4d2a-8c07-f4dbed1d9c89 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:29.483410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e46a8-737a-423f-aa15-57cba25a134c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:29.483702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e46a8-737a-423f-aa15-57cba25a134c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:29.483988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e46a8-737a-423f-aa15-57cba25a134c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:29.490724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e46a8-737a-423f-aa15-57cba25a134c admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:29.490724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e46a8-737a-423f-aa15-57cba25a134c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:29.490724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e46a8-737a-423f-aa15-57cba25a134c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:29.510678 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1281/2554] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:27:29 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:27:29.514887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06394dc7-0fff-431a-a4a7-f92c9fb2edbf admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:29.530715 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06394dc7-0fff-431a-a4a7-f92c9fb2edbf admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:29.536258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3520561-0fd0-4ffe-adf3-0224c43b8d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:29.536891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3520561-0fd0-4ffe-adf3-0224c43b8d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:29.537178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3520561-0fd0-4ffe-adf3-0224c43b8d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:29.537458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3520561-0fd0-4ffe-adf3-0224c43b8d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:29.538169 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3520561-0fd0-4ffe-adf3-0224c43b8d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:29.539901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3520561-0fd0-4ffe-adf3-0224c43b8d10 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:29.552559 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1274/2555] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:29 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:29.556864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e46a8-737a-423f-aa15-57cba25a134c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:29.567603 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e46a8-737a-423f-aa15-57cba25a134c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:29.572406 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a0da7-e660-477b-b319-561561f3e970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:29.573087 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a0da7-e660-477b-b319-561561f3e970 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:29.573373 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a0da7-e660-477b-b319-561561f3e970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:29.573654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a0da7-e660-477b-b319-561561f3e970 admin admin] PATH_INFO: `/v3/projects/658c4f0912ce42b5a1a0b36d22906678/users/5dbe35ffde194827b84288648de1c818/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:29.589655 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a0da7-e660-477b-b319-561561f3e970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=658c4f0912ce42b5a1a0b36d22906678, user_id=5dbe35ffde194827b84288648de1c818, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:29.592397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a0da7-e660-477b-b319-561561f3e970 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:29.606589 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1282/2556] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:27:29 2022] PUT /identity/v3/projects/658c4f0912ce42b5a1a0b36d22906678/users/5dbe35ffde194827b84288648de1c818/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:29.610534 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3520561-0fd0-4ffe-adf3-0224c43b8d10 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:29.621851 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3520561-0fd0-4ffe-adf3-0224c43b8d10 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:29.656305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:29.729624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71296b-feff-47b7-b189-8fea47aacf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:29.731840 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71296b-feff-47b7-b189-8fea47aacf96 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:29.732224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71296b-feff-47b7-b189-8fea47aacf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:29.732704 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71296b-feff-47b7-b189-8fea47aacf96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:29.733411 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71296b-feff-47b7-b189-8fea47aacf96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:29.739045 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71296b-feff-47b7-b189-8fea47aacf96 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:29.757300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1275/2557] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:29 2022] GET /identity/v3/roles => generated 1427 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:29.762258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a0da7-e660-477b-b319-561561f3e970 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:29.768345 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:29.776016 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a0da7-e660-477b-b319-561561f3e970 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:29.782084 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d53b4e-0d0b-4483-92ce-06e5a08ee4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:29.782950 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d53b4e-0d0b-4483-92ce-06e5a08ee4a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:29.783307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d53b4e-0d0b-4483-92ce-06e5a08ee4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:29.783714 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d53b4e-0d0b-4483-92ce-06e5a08ee4a5 admin admin] PATH_INFO: `/v3/projects/658c4f0912ce42b5a1a0b36d22906678/users/5dbe35ffde194827b84288648de1c818/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:29.787917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d53b4e-0d0b-4483-92ce-06e5a08ee4a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=658c4f0912ce42b5a1a0b36d22906678, user_id=5dbe35ffde194827b84288648de1c818, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:29.791145 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d53b4e-0d0b-4483-92ce-06e5a08ee4a5 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:29.805014 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1283/2558] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:27:29 2022] PUT /identity/v3/projects/658c4f0912ce42b5a1a0b36d22906678/users/5dbe35ffde194827b84288648de1c818/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:29.816108 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da92b90-0b2a-447a-aa2e-b2e6244d14c2 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:29.816307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da92b90-0b2a-447a-aa2e-b2e6244d14c2 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:29.816998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da92b90-0b2a-447a-aa2e-b2e6244d14c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:29.849006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5da92b90-0b2a-447a-aa2e-b2e6244d14c2 None None] MFA Rules not processed for user `5dbe35ffde194827b84288648de1c818`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:29.850336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5da92b90-0b2a-447a-aa2e-b2e6244d14c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:29.920049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1276/2559] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:27:29.932103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d53b4e-0d0b-4483-92ce-06e5a08ee4a5 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:29.933902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b7d53b4e-0d0b-4483-92ce-06e5a08ee4a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:29.993097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d53b4e-0d0b-4483-92ce-06e5a08ee4a5 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:29.996790 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e763daa-2604-4a83-800e-180d830dad26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:29.997411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e763daa-2604-4a83-800e-180d830dad26 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:29.997697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e763daa-2604-4a83-800e-180d830dad26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:29.997984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e763daa-2604-4a83-800e-180d830dad26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:30.009041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e763daa-2604-4a83-800e-180d830dad26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:30.010533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e763daa-2604-4a83-800e-180d830dad26 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:30.021438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1284/2560] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:30.089416 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da92b90-0b2a-447a-aa2e-b2e6244d14c2 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:30.153932 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da92b90-0b2a-447a-aa2e-b2e6244d14c2 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:30.158529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e06e7d-6d1a-48ac-801c-2cf6732b7151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:30.159205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e06e7d-6d1a-48ac-801c-2cf6732b7151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:30.159491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e06e7d-6d1a-48ac-801c-2cf6732b7151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:30.159771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e06e7d-6d1a-48ac-801c-2cf6732b7151 admin admin] PATH_INFO: `/v3/users/88fe6eb7ca7b48eea0a3555c6b36c31f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:30.160963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e06e7d-6d1a-48ac-801c-2cf6732b7151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88fe6eb7ca7b48eea0a3555c6b36c31f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:30.162943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e06e7d-6d1a-48ac-801c-2cf6732b7151 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:30.214216 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e5e06e7d-6d1a-48ac-801c-2cf6732b7151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88fe6eb7ca7b48eea0a3555c6b36c31f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:30.215975 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:30.222812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e5e06e7d-6d1a-48ac-801c-2cf6732b7151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88fe6eb7ca7b48eea0a3555c6b36c31f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:30.223613 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e5e06e7d-6d1a-48ac-801c-2cf6732b7151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88fe6eb7ca7b48eea0a3555c6b36c31f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:30.224220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e5e06e7d-6d1a-48ac-801c-2cf6732b7151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88fe6eb7ca7b48eea0a3555c6b36c31f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:30.236227 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-e5e06e7d-6d1a-48ac-801c-2cf6732b7151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88fe6eb7ca7b48eea0a3555c6b36c31f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:30.249009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1277/2561] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:27:30 2022] DELETE /identity/v3/users/88fe6eb7ca7b48eea0a3555c6b36c31f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:30.254026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7e5f-46ba-4908-b206-00440d6e1db7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:30.254347 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7e5f-46ba-4908-b206-00440d6e1db7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:30.254626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7e5f-46ba-4908-b206-00440d6e1db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:30.274223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e80e7e5f-46ba-4908-b206-00440d6e1db7 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:30.275492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e80e7e5f-46ba-4908-b206-00440d6e1db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:30.334616 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1285/2562] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:30.344469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e06e7d-6d1a-48ac-801c-2cf6732b7151 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:30.345923 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e5e06e7d-6d1a-48ac-801c-2cf6732b7151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:30.402373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e06e7d-6d1a-48ac-801c-2cf6732b7151 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:30.405980 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe92d1-b9b6-4c74-9eb1-119a83c704f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:30.406587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe92d1-b9b6-4c74-9eb1-119a83c704f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:30.406912 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe92d1-b9b6-4c74-9eb1-119a83c704f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:30.407184 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe92d1-b9b6-4c74-9eb1-119a83c704f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:30.416157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe92d1-b9b6-4c74-9eb1-119a83c704f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:30.417585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe92d1-b9b6-4c74-9eb1-119a83c704f0 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:30.427220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1278/2563] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:30.728174 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e7e5f-46ba-4908-b206-00440d6e1db7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:30.729693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e80e7e5f-46ba-4908-b206-00440d6e1db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:30.743698 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e7e5f-46ba-4908-b206-00440d6e1db7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:30.748721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:30.750005 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:30.750547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:30.751090 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] PATH_INFO: `/v3/projects/e243242e5bfb46b48f2808db58ed8cbd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:30.753051 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e243242e5bfb46b48f2808db58ed8cbd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:30.756197 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:30.811379 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e243242e5bfb46b48f2808db58ed8cbd', 'request_id': 'req-75b29c6f-2213-4efb-b704-92dd8adb9935'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:30.824281 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e243242e5bfb46b48f2808db58ed8cbd', 'request_id': 'req-75b29c6f-2213-4efb-b704-92dd8adb9935'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:30.834238 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e243242e5bfb46b48f2808db58ed8cbd', 'request_id': 'req-75b29c6f-2213-4efb-b704-92dd8adb9935'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:30.850339 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] The token cache is being invalidate because project e243242e5bfb46b48f2808db58ed8cbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:30.850810 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:30.851634 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:30.853187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1286/2564] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:27:30 2022] DELETE /identity/v3/projects/e243242e5bfb46b48f2808db58ed8cbd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:30.857417 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe92d1-b9b6-4c74-9eb1-119a83c704f0 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:30.868042 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe92d1-b9b6-4c74-9eb1-119a83c704f0 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:30.932670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ceeda-d744-422b-8a68-e98dc15b23b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:30.933314 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ceeda-d744-422b-8a68-e98dc15b23b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:30.933594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ceeda-d744-422b-8a68-e98dc15b23b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:30.933872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ceeda-d744-422b-8a68-e98dc15b23b8 admin admin] PATH_INFO: `/v3/users/5dbe35ffde194827b84288648de1c818` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:30.935090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ceeda-d744-422b-8a68-e98dc15b23b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dbe35ffde194827b84288648de1c818)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:30.937020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ceeda-d744-422b-8a68-e98dc15b23b8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:31.003022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ee8ceeda-d744-422b-8a68-e98dc15b23b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dbe35ffde194827b84288648de1c818'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:31.011653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ee8ceeda-d744-422b-8a68-e98dc15b23b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dbe35ffde194827b84288648de1c818'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:31.012287 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ee8ceeda-d744-422b-8a68-e98dc15b23b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dbe35ffde194827b84288648de1c818'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:31.012897 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ee8ceeda-d744-422b-8a68-e98dc15b23b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dbe35ffde194827b84288648de1c818'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:31.026195 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:31.027683 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ee8ceeda-d744-422b-8a68-e98dc15b23b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dbe35ffde194827b84288648de1c818'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:31.043618 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1279/2565] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:27:30 2022] DELETE /identity/v3/users/5dbe35ffde194827b84288648de1c818 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:27:31.354965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:31.356419 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:31.373130 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b29c6f-2213-4efb-b704-92dd8adb9935 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:31.394520 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:31.439549 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a05983-5b46-4ead-a62a-84c1ccaef153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:31.440285 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a05983-5b46-4ead-a62a-84c1ccaef153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:31.440787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a05983-5b46-4ead-a62a-84c1ccaef153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:31.441296 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a05983-5b46-4ead-a62a-84c1ccaef153 admin admin] PATH_INFO: `/v3/projects/658c4f0912ce42b5a1a0b36d22906678` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:31.443054 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a05983-5b46-4ead-a62a-84c1ccaef153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=658c4f0912ce42b5a1a0b36d22906678)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:31.445298 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a05983-5b46-4ead-a62a-84c1ccaef153 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:31.507847 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3a05983-5b46-4ead-a62a-84c1ccaef153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658c4f0912ce42b5a1a0b36d22906678', 'request_id': 'req-c3a05983-5b46-4ead-a62a-84c1ccaef153'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:31.516938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3a05983-5b46-4ead-a62a-84c1ccaef153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658c4f0912ce42b5a1a0b36d22906678', 'request_id': 'req-c3a05983-5b46-4ead-a62a-84c1ccaef153'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:31.522904 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3a05983-5b46-4ead-a62a-84c1ccaef153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658c4f0912ce42b5a1a0b36d22906678', 'request_id': 'req-c3a05983-5b46-4ead-a62a-84c1ccaef153'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:31.536873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3a05983-5b46-4ead-a62a-84c1ccaef153 admin admin] The token cache is being invalidate because project 658c4f0912ce42b5a1a0b36d22906678 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:31.537190 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3a05983-5b46-4ead-a62a-84c1ccaef153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:31.537858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-c3a05983-5b46-4ead-a62a-84c1ccaef153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:31.539515 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1287/2566] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:27:31 2022] DELETE /identity/v3/projects/658c4f0912ce42b5a1a0b36d22906678 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:31.544432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55753664-0e6d-4f97-8f1b-438318840688 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:31.544730 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55753664-0e6d-4f97-8f1b-438318840688 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:31.545011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55753664-0e6d-4f97-8f1b-438318840688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:31.565725 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-55753664-0e6d-4f97-8f1b-438318840688 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:31.566999 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-55753664-0e6d-4f97-8f1b-438318840688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:31.629707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1280/2567] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:31.639217 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802fb92-f687-4ed8-bdbf-99ce221a3cd6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:31.639512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802fb92-f687-4ed8-bdbf-99ce221a3cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:31.639794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802fb92-f687-4ed8-bdbf-99ce221a3cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:31.660993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-c802fb92-f687-4ed8-bdbf-99ce221a3cd6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:31.662131 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c802fb92-f687-4ed8-bdbf-99ce221a3cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:31.670133 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1288/2568] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:31.675336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55753664-0e6d-4f97-8f1b-438318840688 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:31.685424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55753664-0e6d-4f97-8f1b-438318840688 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:31.690938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30f2b3-ebf2-421b-8fad-900f26aba955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:31.691582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30f2b3-ebf2-421b-8fad-900f26aba955 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:31.691996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30f2b3-ebf2-421b-8fad-900f26aba955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:31.692206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30f2b3-ebf2-421b-8fad-900f26aba955 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:31.692963 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30f2b3-ebf2-421b-8fad-900f26aba955 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:27:31.693280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30f2b3-ebf2-421b-8fad-900f26aba955 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:31.695787 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30f2b3-ebf2-421b-8fad-900f26aba955 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:31.708255 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1281/2569] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:27:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:27:31.712825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802fb92-f687-4ed8-bdbf-99ce221a3cd6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:31.725646 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802fb92-f687-4ed8-bdbf-99ce221a3cd6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:31.733107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182759b-80ef-4995-bb7d-b30f98ed5ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:31.733746 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182759b-80ef-4995-bb7d-b30f98ed5ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:31.734039 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182759b-80ef-4995-bb7d-b30f98ed5ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:31.734390 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182759b-80ef-4995-bb7d-b30f98ed5ca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:31.735194 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182759b-80ef-4995-bb7d-b30f98ed5ca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:31.738064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182759b-80ef-4995-bb7d-b30f98ed5ca3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:31.771482 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1289/2570] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:27:31 2022] POST /identity/v3/projects => generated 400 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:27:31.777280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30f2b3-ebf2-421b-8fad-900f26aba955 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:31.789039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30f2b3-ebf2-421b-8fad-900f26aba955 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:31.852865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570b474-2678-4af3-8e85-4a7830047d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:31.853483 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570b474-2678-4af3-8e85-4a7830047d62 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:31.853761 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570b474-2678-4af3-8e85-4a7830047d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:31.854033 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570b474-2678-4af3-8e85-4a7830047d62 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:31.854723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570b474-2678-4af3-8e85-4a7830047d62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:31.856654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570b474-2678-4af3-8e85-4a7830047d62 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:31.878346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1282/2571] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:27:31 2022] POST /identity/v3/users => generated 301 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:27:31.882597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182759b-80ef-4995-bb7d-b30f98ed5ca3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:31.893050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182759b-80ef-4995-bb7d-b30f98ed5ca3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:31.897549 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c07d51-b87f-4f6d-8ab1-08e05c7b98c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:31.898183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c07d51-b87f-4f6d-8ab1-08e05c7b98c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:31.898460 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c07d51-b87f-4f6d-8ab1-08e05c7b98c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:31.898761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c07d51-b87f-4f6d-8ab1-08e05c7b98c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:31.899446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c07d51-b87f-4f6d-8ab1-08e05c7b98c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:31.901040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c07d51-b87f-4f6d-8ab1-08e05c7b98c3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:31.911821 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1290/2572] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:31 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:31.915745 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570b474-2678-4af3-8e85-4a7830047d62 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:31.926521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570b474-2678-4af3-8e85-4a7830047d62 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:31.931127 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cc63c-3e9d-40d4-bfdb-4e5227f535d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:31.931734 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cc63c-3e9d-40d4-bfdb-4e5227f535d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:31.932014 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cc63c-3e9d-40d4-bfdb-4e5227f535d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:31.932287 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cc63c-3e9d-40d4-bfdb-4e5227f535d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:31.932977 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cc63c-3e9d-40d4-bfdb-4e5227f535d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:31.934563 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cc63c-3e9d-40d4-bfdb-4e5227f535d2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:31.947021 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1283/2573] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:31 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:27:31.951526 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c07d51-b87f-4f6d-8ab1-08e05c7b98c3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:31.962442 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c07d51-b87f-4f6d-8ab1-08e05c7b98c3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:31.967236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5cd4a-2a64-4178-962a-f5e3ef1a675a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:31.967906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5cd4a-2a64-4178-962a-f5e3ef1a675a admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:31.968187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5cd4a-2a64-4178-962a-f5e3ef1a675a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:31.968466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5cd4a-2a64-4178-962a-f5e3ef1a675a admin admin] PATH_INFO: `/v3/projects/994eccb521b548cda90e1fa18ac45ad5/users/ca73462d09574b3894b22195f661f7a5/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:31.984811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5cd4a-2a64-4178-962a-f5e3ef1a675a admin admin] RBAC: Authorizing `identity:create_grant(project_id=994eccb521b548cda90e1fa18ac45ad5, user_id=ca73462d09574b3894b22195f661f7a5, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:31.987649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5cd4a-2a64-4178-962a-f5e3ef1a675a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:32.006445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1291/2574] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:27:31 2022] PUT /identity/v3/projects/994eccb521b548cda90e1fa18ac45ad5/users/ca73462d09574b3894b22195f661f7a5/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:32.019264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f6723-951d-4571-ba72-3a20e3dd5452 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:32.019585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f6723-951d-4571-ba72-3a20e3dd5452 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:32.019866 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f6723-951d-4571-ba72-3a20e3dd5452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:32.050436 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2b3f6723-951d-4571-ba72-3a20e3dd5452 None None] MFA Rules not processed for user `ca73462d09574b3894b22195f661f7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:32.051705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2b3f6723-951d-4571-ba72-3a20e3dd5452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:32.096751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:32.120562 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1284/2575] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:32 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:32.125179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5cd4a-2a64-4178-962a-f5e3ef1a675a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:32.172612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:32.192234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5cd4a-2a64-4178-962a-f5e3ef1a675a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:32.197718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdb182-6821-48eb-8634-539dc90635c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:32.198447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdb182-6821-48eb-8634-539dc90635c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:32.198758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdb182-6821-48eb-8634-539dc90635c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:32.199039 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdb182-6821-48eb-8634-539dc90635c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:32.199725 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdb182-6821-48eb-8634-539dc90635c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:32.201906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cdb182-6821-48eb-8634-539dc90635c0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:32.228997 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1292/2576] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:27:32 2022] POST /identity/v3/projects => generated 402 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:27:32.233186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f6723-951d-4571-ba72-3a20e3dd5452 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:32.244120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f6723-951d-4571-ba72-3a20e3dd5452 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:32.304937 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c96d7b-9f9b-437e-8f7c-eb13aab9ba4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:32.305582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c96d7b-9f9b-437e-8f7c-eb13aab9ba4d admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:32.305869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c96d7b-9f9b-437e-8f7c-eb13aab9ba4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:32.306151 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c96d7b-9f9b-437e-8f7c-eb13aab9ba4d admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:32.306851 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c96d7b-9f9b-437e-8f7c-eb13aab9ba4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:32.308794 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c96d7b-9f9b-437e-8f7c-eb13aab9ba4d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:32.331621 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1285/2577] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:27:32 2022] POST /identity/v3/users => generated 308 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:27:32.335669 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdb182-6821-48eb-8634-539dc90635c0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:32.346437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdb182-6821-48eb-8634-539dc90635c0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:32.351086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0312c49-1d85-46a6-bf91-4448a8f6319a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:32.351693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0312c49-1d85-46a6-bf91-4448a8f6319a admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:32.351974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0312c49-1d85-46a6-bf91-4448a8f6319a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:32.352249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0312c49-1d85-46a6-bf91-4448a8f6319a admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:32.352938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0312c49-1d85-46a6-bf91-4448a8f6319a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:32.354529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0312c49-1d85-46a6-bf91-4448a8f6319a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:32.365534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1293/2578] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:32 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:32.369524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c96d7b-9f9b-437e-8f7c-eb13aab9ba4d admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:32.379705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c96d7b-9f9b-437e-8f7c-eb13aab9ba4d admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:32.384616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a71aa-3ffc-43da-891d-00eccf10d3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:32.385281 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a71aa-3ffc-43da-891d-00eccf10d3b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:32.385573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a71aa-3ffc-43da-891d-00eccf10d3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:32.385849 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a71aa-3ffc-43da-891d-00eccf10d3b3 admin admin] PATH_INFO: `/v3/projects/bc130f63bc964a1285b9ab8ad674884e/users/60934d79c05e434b9a105c4e202e7542/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:32.401250 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a71aa-3ffc-43da-891d-00eccf10d3b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc130f63bc964a1285b9ab8ad674884e, user_id=60934d79c05e434b9a105c4e202e7542, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:32.403961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a71aa-3ffc-43da-891d-00eccf10d3b3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:32.417807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1286/2579] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:27:32 2022] PUT /identity/v3/projects/bc130f63bc964a1285b9ab8ad674884e/users/60934d79c05e434b9a105c4e202e7542/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:32.422061 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0312c49-1d85-46a6-bf91-4448a8f6319a admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:32.433178 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0312c49-1d85-46a6-bf91-4448a8f6319a admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:32.489563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd509c-be70-4f3e-861b-b042fe81f5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:32.490173 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd509c-be70-4f3e-861b-b042fe81f5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:32.490455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd509c-be70-4f3e-861b-b042fe81f5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:32.490758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd509c-be70-4f3e-861b-b042fe81f5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:32.491449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd509c-be70-4f3e-861b-b042fe81f5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:32.493044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd509c-be70-4f3e-861b-b042fe81f5c8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:32.502694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1294/2580] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:32 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:32.506852 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a71aa-3ffc-43da-891d-00eccf10d3b3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:32.516985 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a71aa-3ffc-43da-891d-00eccf10d3b3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:32.523024 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6685213-b296-4d63-87a6-819a0a90cc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:32.523695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6685213-b296-4d63-87a6-819a0a90cc83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:32.523978 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6685213-b296-4d63-87a6-819a0a90cc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:32.524257 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6685213-b296-4d63-87a6-819a0a90cc83 admin admin] PATH_INFO: `/v3/projects/bc130f63bc964a1285b9ab8ad674884e/users/60934d79c05e434b9a105c4e202e7542/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:32.526278 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6685213-b296-4d63-87a6-819a0a90cc83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc130f63bc964a1285b9ab8ad674884e, user_id=60934d79c05e434b9a105c4e202e7542, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:32.528961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6685213-b296-4d63-87a6-819a0a90cc83 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:32.542890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1287/2581] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:27:32 2022] PUT /identity/v3/projects/bc130f63bc964a1285b9ab8ad674884e/users/60934d79c05e434b9a105c4e202e7542/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:32.553569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e460d0-29f0-44d0-8ab0-fa82a3abb5d1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:32.553860 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e460d0-29f0-44d0-8ab0-fa82a3abb5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:32.554140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e460d0-29f0-44d0-8ab0-fa82a3abb5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:32.583733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-50e460d0-29f0-44d0-8ab0-fa82a3abb5d1 None None] MFA Rules not processed for user `60934d79c05e434b9a105c4e202e7542`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:32.584988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-50e460d0-29f0-44d0-8ab0-fa82a3abb5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:32.657022 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1295/2582] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:32 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:32.667995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6685213-b296-4d63-87a6-819a0a90cc83 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:32.670112 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f6685213-b296-4d63-87a6-819a0a90cc83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:32.747846 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6685213-b296-4d63-87a6-819a0a90cc83 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:32.751947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acacc0-4849-413e-a948-974292300294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:32.752566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acacc0-4849-413e-a948-974292300294 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:32.752845 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acacc0-4849-413e-a948-974292300294 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:32.753119 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acacc0-4849-413e-a948-974292300294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:32.764196 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acacc0-4849-413e-a948-974292300294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:32.765660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acacc0-4849-413e-a948-974292300294 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:32.777954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1288/2583] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:32 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:32.779725 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e460d0-29f0-44d0-8ab0-fa82a3abb5d1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:32.781160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-50e460d0-29f0-44d0-8ab0-fa82a3abb5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:32.826906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:32.847019 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e460d0-29f0-44d0-8ab0-fa82a3abb5d1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:32.850857 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a1d5c-454f-46d8-b601-5a41d44203e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:32.851492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a1d5c-454f-46d8-b601-5a41d44203e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:32.851776 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a1d5c-454f-46d8-b601-5a41d44203e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:32.852073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a1d5c-454f-46d8-b601-5a41d44203e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:32.853837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3b4a1d5c-454f-46d8-b601-5a41d44203e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:32.857673 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acacc0-4849-413e-a948-974292300294 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:32.867600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acacc0-4849-413e-a948-974292300294 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:32.871230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0b014-ad0a-4280-b7fe-5788b513ef48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:32.871826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0b014-ad0a-4280-b7fe-5788b513ef48 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:32.872107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0b014-ad0a-4280-b7fe-5788b513ef48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:32.872378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0b014-ad0a-4280-b7fe-5788b513ef48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:32.907722 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:32.921610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a1d5c-454f-46d8-b601-5a41d44203e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:32.923121 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a1d5c-454f-46d8-b601-5a41d44203e3 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:32.932945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0b014-ad0a-4280-b7fe-5788b513ef48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:32.934161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1296/2584] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:32.934548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0b014-ad0a-4280-b7fe-5788b513ef48 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:32.944770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1289/2585] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:32 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:27:35.893351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a1d5c-454f-46d8-b601-5a41d44203e3 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:35.894996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3b4a1d5c-454f-46d8-b601-5a41d44203e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:35.989584 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a1d5c-454f-46d8-b601-5a41d44203e3 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:35.997155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041e112-843a-46f0-b9f5-d10d4dd4b3f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:35.998706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041e112-843a-46f0-b9f5-d10d4dd4b3f8 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:35.998706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041e112-843a-46f0-b9f5-d10d4dd4b3f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:35.998706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041e112-843a-46f0-b9f5-d10d4dd4b3f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:36.008423 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5041e112-843a-46f0-b9f5-d10d4dd4b3f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:36.025627 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041e112-843a-46f0-b9f5-d10d4dd4b3f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:36.027488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041e112-843a-46f0-b9f5-d10d4dd4b3f8 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:36.039816 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1297/2586] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:37.179102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0b014-ad0a-4280-b7fe-5788b513ef48 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:37.190218 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0b014-ad0a-4280-b7fe-5788b513ef48 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:37.195206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971140f-db70-4d3b-9f61-a1c989ff0d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:37.195869 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971140f-db70-4d3b-9f61-a1c989ff0d5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:37.196153 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971140f-db70-4d3b-9f61-a1c989ff0d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:37.196430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971140f-db70-4d3b-9f61-a1c989ff0d5b admin admin] PATH_INFO: `/v3/users/ca73462d09574b3894b22195f661f7a5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:37.197603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971140f-db70-4d3b-9f61-a1c989ff0d5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca73462d09574b3894b22195f661f7a5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:37.199599 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971140f-db70-4d3b-9f61-a1c989ff0d5b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:37.250054 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d971140f-db70-4d3b-9f61-a1c989ff0d5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca73462d09574b3894b22195f661f7a5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.259389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d971140f-db70-4d3b-9f61-a1c989ff0d5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca73462d09574b3894b22195f661f7a5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.260134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d971140f-db70-4d3b-9f61-a1c989ff0d5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca73462d09574b3894b22195f661f7a5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.260792 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d971140f-db70-4d3b-9f61-a1c989ff0d5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca73462d09574b3894b22195f661f7a5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.276094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d971140f-db70-4d3b-9f61-a1c989ff0d5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca73462d09574b3894b22195f661f7a5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.290474 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1290/2587] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:27:37 2022] DELETE /identity/v3/users/ca73462d09574b3894b22195f661f7a5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:37.295201 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f37ba2-e65c-4a10-aa19-3f0f390bdba1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:37.295715 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f37ba2-e65c-4a10-aa19-3f0f390bdba1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:37.295814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f37ba2-e65c-4a10-aa19-3f0f390bdba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:37.315690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-40f37ba2-e65c-4a10-aa19-3f0f390bdba1 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:37.316927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-40f37ba2-e65c-4a10-aa19-3f0f390bdba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:37.381775 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1298/2588] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:37.391900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971140f-db70-4d3b-9f61-a1c989ff0d5b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:37.393900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d971140f-db70-4d3b-9f61-a1c989ff0d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:37.454026 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971140f-db70-4d3b-9f61-a1c989ff0d5b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:37.458402 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cf4e1-0682-405d-9eb9-228c65df4f4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:37.459089 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cf4e1-0682-405d-9eb9-228c65df4f4f service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:37.459378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cf4e1-0682-405d-9eb9-228c65df4f4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:37.459661 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cf4e1-0682-405d-9eb9-228c65df4f4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:37.469693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cf4e1-0682-405d-9eb9-228c65df4f4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:37.471757 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2cf4e1-0682-405d-9eb9-228c65df4f4f service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:37.485785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1291/2589] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:37.791820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f37ba2-e65c-4a10-aa19-3f0f390bdba1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:37.804212 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-40f37ba2-e65c-4a10-aa19-3f0f390bdba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:37.826931 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f37ba2-e65c-4a10-aa19-3f0f390bdba1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:37.840839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:37.841757 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:37.842115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:37.842464 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] PATH_INFO: `/v3/projects/994eccb521b548cda90e1fa18ac45ad5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:37.843870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=994eccb521b548cda90e1fa18ac45ad5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:37.847559 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:37.863031 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:37.911976 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '994eccb521b548cda90e1fa18ac45ad5', 'request_id': 'req-fa51a966-7839-49a1-ad2b-f9a38ee98576'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.929988 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '994eccb521b548cda90e1fa18ac45ad5', 'request_id': 'req-fa51a966-7839-49a1-ad2b-f9a38ee98576'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.939339 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '994eccb521b548cda90e1fa18ac45ad5', 'request_id': 'req-fa51a966-7839-49a1-ad2b-f9a38ee98576'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.963702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] The token cache is being invalidate because project 994eccb521b548cda90e1fa18ac45ad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:37.964151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.965584 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:37.968335 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1299/2590] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:27:37 2022] DELETE /identity/v3/projects/994eccb521b548cda90e1fa18ac45ad5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:37.973313 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cf4e1-0682-405d-9eb9-228c65df4f4f service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:37.990445 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2cf4e1-0682-405d-9eb9-228c65df4f4f service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:38.064068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:38.080043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ad1d8-0788-4cae-a7ce-71c54b45986a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:38.080743 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ad1d8-0788-4cae-a7ce-71c54b45986a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:38.081049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ad1d8-0788-4cae-a7ce-71c54b45986a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:38.082896 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ad1d8-0788-4cae-a7ce-71c54b45986a admin admin] PATH_INFO: `/v3/users/60934d79c05e434b9a105c4e202e7542` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:38.085993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ad1d8-0788-4cae-a7ce-71c54b45986a admin admin] RBAC: Authorizing `identity:delete_user(user_id=60934d79c05e434b9a105c4e202e7542)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:38.088012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ad1d8-0788-4cae-a7ce-71c54b45986a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:38.155330 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cc7ad1d8-0788-4cae-a7ce-71c54b45986a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60934d79c05e434b9a105c4e202e7542'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.165681 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cc7ad1d8-0788-4cae-a7ce-71c54b45986a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60934d79c05e434b9a105c4e202e7542'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.166500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cc7ad1d8-0788-4cae-a7ce-71c54b45986a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60934d79c05e434b9a105c4e202e7542'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.167259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cc7ad1d8-0788-4cae-a7ce-71c54b45986a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60934d79c05e434b9a105c4e202e7542'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.185529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cc7ad1d8-0788-4cae-a7ce-71c54b45986a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60934d79c05e434b9a105c4e202e7542'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.200110 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1292/2591] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:27:37 2022] DELETE /identity/v3/users/60934d79c05e434b9a105c4e202e7542 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:27:38.513445 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:38.515546 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:38.535552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51a966-7839-49a1-ad2b-f9a38ee98576 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:38.621150 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:38.621813 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:38.622098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:38.622374 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4 admin admin] PATH_INFO: `/v3/projects/bc130f63bc964a1285b9ab8ad674884e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:38.623721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc130f63bc964a1285b9ab8ad674884e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:38.625718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:38.683479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc130f63bc964a1285b9ab8ad674884e', 'request_id': 'req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.692286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc130f63bc964a1285b9ab8ad674884e', 'request_id': 'req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.707492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc130f63bc964a1285b9ab8ad674884e', 'request_id': 'req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.721050 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4 admin admin] The token cache is being invalidate because project bc130f63bc964a1285b9ab8ad674884e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:38.721338 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.721948 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-f163cc3e-bdb0-43b6-8efc-ccf1ecaa9df4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:38.723879 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1300/2592] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:27:38 2022] DELETE /identity/v3/projects/bc130f63bc964a1285b9ab8ad674884e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:38.728425 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21c513-777b-4d23-b926-f6ce13171838 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:38.728719 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21c513-777b-4d23-b926-f6ce13171838 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:38.729019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21c513-777b-4d23-b926-f6ce13171838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:38.749707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9f21c513-777b-4d23-b926-f6ce13171838 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:38.751105 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9f21c513-777b-4d23-b926-f6ce13171838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:38.818204 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1293/2593] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:38.827307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c7a35-7152-4114-a5a6-70bcc945b869 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:38.827614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c7a35-7152-4114-a5a6-70bcc945b869 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:38.827894 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c7a35-7152-4114-a5a6-70bcc945b869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:38.848344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-a87c7a35-7152-4114-a5a6-70bcc945b869 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:38.849582 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a87c7a35-7152-4114-a5a6-70bcc945b869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:38.858632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1301/2594] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:38.863182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21c513-777b-4d23-b926-f6ce13171838 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:38.873289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21c513-777b-4d23-b926-f6ce13171838 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:38.879429 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca66355-69a8-45a7-a88d-2a7f973712ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:38.880062 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca66355-69a8-45a7-a88d-2a7f973712ce admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:38.880340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca66355-69a8-45a7-a88d-2a7f973712ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:38.880613 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca66355-69a8-45a7-a88d-2a7f973712ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:38.881317 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca66355-69a8-45a7-a88d-2a7f973712ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:27:38.881622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca66355-69a8-45a7-a88d-2a7f973712ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:38.883301 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca66355-69a8-45a7-a88d-2a7f973712ce admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:38.896654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1294/2595] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:27:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:27:38.900760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c7a35-7152-4114-a5a6-70bcc945b869 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:38.910243 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c7a35-7152-4114-a5a6-70bcc945b869 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:38.922364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f87cf-e25a-476f-a4a5-96d894580ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:38.923864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f87cf-e25a-476f-a4a5-96d894580ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:38.924229 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f87cf-e25a-476f-a4a5-96d894580ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:38.924584 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f87cf-e25a-476f-a4a5-96d894580ed2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:38.925351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f87cf-e25a-476f-a4a5-96d894580ed2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:38.927435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f87cf-e25a-476f-a4a5-96d894580ed2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:38.964069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1302/2596] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:27:38 2022] POST /identity/v3/projects => generated 368 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:27:38.968467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca66355-69a8-45a7-a88d-2a7f973712ce admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:38.978587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca66355-69a8-45a7-a88d-2a7f973712ce admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:39.036383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:39.054519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d57ab-5cf8-45db-8021-92dbb55ab0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:39.055322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d57ab-5cf8-45db-8021-92dbb55ab0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:39.055779 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d57ab-5cf8-45db-8021-92dbb55ab0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:39.056163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d57ab-5cf8-45db-8021-92dbb55ab0e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:39.056971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d57ab-5cf8-45db-8021-92dbb55ab0e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:39.060286 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1d57ab-5cf8-45db-8021-92dbb55ab0e2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:39.093613 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1295/2597] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:27:38 2022] POST /identity/v3/users => generated 285 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:27:39.097968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f87cf-e25a-476f-a4a5-96d894580ed2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:39.103892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:39.112395 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f87cf-e25a-476f-a4a5-96d894580ed2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:39.119511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd130f-2ea6-49a4-ae45-5f21ee115b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:39.120153 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd130f-2ea6-49a4-ae45-5f21ee115b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:39.120569 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd130f-2ea6-49a4-ae45-5f21ee115b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:39.120855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd130f-2ea6-49a4-ae45-5f21ee115b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:39.122046 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd130f-2ea6-49a4-ae45-5f21ee115b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:39.123727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd130f-2ea6-49a4-ae45-5f21ee115b37 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:39.135632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1303/2598] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:39 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:39.140831 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d57ab-5cf8-45db-8021-92dbb55ab0e2 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:39.160004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d57ab-5cf8-45db-8021-92dbb55ab0e2 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:39.167910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ba6a5-5a55-43c4-a188-573c427fcf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:39.168554 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ba6a5-5a55-43c4-a188-573c427fcf1e admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:39.168898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ba6a5-5a55-43c4-a188-573c427fcf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:39.169183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ba6a5-5a55-43c4-a188-573c427fcf1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:39.169899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ba6a5-5a55-43c4-a188-573c427fcf1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:39.173524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ba6a5-5a55-43c4-a188-573c427fcf1e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:39.191270 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1296/2599] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:39 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:39.197450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd130f-2ea6-49a4-ae45-5f21ee115b37 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:39.209049 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd130f-2ea6-49a4-ae45-5f21ee115b37 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:39.241677 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0a7f8-13bd-453a-9c59-dde459bf0d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:39.242769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0a7f8-13bd-453a-9c59-dde459bf0d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:39.243077 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0a7f8-13bd-453a-9c59-dde459bf0d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:39.243733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0a7f8-13bd-453a-9c59-dde459bf0d89 admin admin] PATH_INFO: `/v3/projects/cf6a4011eeac4641b96fba21ce06d55a/users/6c570d965ffa49c69919e9cc08c71b87/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:39.261363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0a7f8-13bd-453a-9c59-dde459bf0d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf6a4011eeac4641b96fba21ce06d55a, user_id=6c570d965ffa49c69919e9cc08c71b87, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:39.264087 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0a7f8-13bd-453a-9c59-dde459bf0d89 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:39.286725 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1304/2600] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:27:39 2022] PUT /identity/v3/projects/cf6a4011eeac4641b96fba21ce06d55a/users/6c570d965ffa49c69919e9cc08c71b87/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:39.356311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdf87d-2693-4c48-b0bc-ec897d1fbe6b None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:39.356609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdf87d-2693-4c48-b0bc-ec897d1fbe6b None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:39.356927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cdf87d-2693-4c48-b0bc-ec897d1fbe6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:39.396949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-d1cdf87d-2693-4c48-b0bc-ec897d1fbe6b None None] MFA Rules not processed for user `6c570d965ffa49c69919e9cc08c71b87`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:39.398409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d1cdf87d-2693-4c48-b0bc-ec897d1fbe6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:39.484947 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0a7f8-13bd-453a-9c59-dde459bf0d89 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:39.486683 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1297/2601] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:39.487799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-c5a0a7f8-13bd-453a-9c59-dde459bf0d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:39.492522 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdf87d-2693-4c48-b0bc-ec897d1fbe6b None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:39.573024 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cdf87d-2693-4c48-b0bc-ec897d1fbe6b None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:39.573272 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0a7f8-13bd-453a-9c59-dde459bf0d89 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:39.578720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e47474-0ba1-4682-b845-8d197d74836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:39.579022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1db74d-d946-40d2-8b3d-f8e6124c5af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:39.579447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e47474-0ba1-4682-b845-8d197d74836b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:39.579681 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1db74d-d946-40d2-8b3d-f8e6124c5af4 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:39.579843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e47474-0ba1-4682-b845-8d197d74836b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:39.579974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1db74d-d946-40d2-8b3d-f8e6124c5af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:39.580181 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e47474-0ba1-4682-b845-8d197d74836b admin admin] PATH_INFO: `/v3/users/c6795a827b8d411aaa165db7b38435ce` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:39.580351 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1db74d-d946-40d2-8b3d-f8e6124c5af4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:39.580905 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1db74d-d946-40d2-8b3d-f8e6124c5af4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:39.581342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e47474-0ba1-4682-b845-8d197d74836b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6795a827b8d411aaa165db7b38435ce)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:39.583136 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1db74d-d946-40d2-8b3d-f8e6124c5af4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:39.588309 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e47474-0ba1-4682-b845-8d197d74836b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:39.611918 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1298/2602] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:27:39 2022] POST /identity/v3/projects => generated 366 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:27:39.616551 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1db74d-d946-40d2-8b3d-f8e6124c5af4 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:39.639992 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e4e47474-0ba1-4682-b845-8d197d74836b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6795a827b8d411aaa165db7b38435ce'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:39.640658 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e4e47474-0ba1-4682-b845-8d197d74836b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6795a827b8d411aaa165db7b38435ce'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:39.648310 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e4e47474-0ba1-4682-b845-8d197d74836b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6795a827b8d411aaa165db7b38435ce'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:39.648961 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e4e47474-0ba1-4682-b845-8d197d74836b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6795a827b8d411aaa165db7b38435ce'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:39.661615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-e4e47474-0ba1-4682-b845-8d197d74836b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6795a827b8d411aaa165db7b38435ce'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:39.670607 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:39.676275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1305/2603] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:27:39 2022] DELETE /identity/v3/users/c6795a827b8d411aaa165db7b38435ce => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:27:39.685394 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1db74d-d946-40d2-8b3d-f8e6124c5af4 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:39.743765 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64054f0-6f16-421c-94a7-0d47a93cf796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:39.744585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64054f0-6f16-421c-94a7-0d47a93cf796 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:39.744841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64054f0-6f16-421c-94a7-0d47a93cf796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:39.745126 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64054f0-6f16-421c-94a7-0d47a93cf796 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:39.745808 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64054f0-6f16-421c-94a7-0d47a93cf796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:39.747854 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64054f0-6f16-421c-94a7-0d47a93cf796 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:39.769521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1299/2604] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:27:39 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:27:39.774158 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e47474-0ba1-4682-b845-8d197d74836b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:39.775639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e4e47474-0ba1-4682-b845-8d197d74836b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:39.788973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e47474-0ba1-4682-b845-8d197d74836b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:39.793679 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bffa0-9768-4e3e-bc0d-8d1f55185420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:39.794303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bffa0-9768-4e3e-bc0d-8d1f55185420 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:39.794586 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bffa0-9768-4e3e-bc0d-8d1f55185420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:39.794897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bffa0-9768-4e3e-bc0d-8d1f55185420 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:39.795600 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bffa0-9768-4e3e-bc0d-8d1f55185420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:39.797209 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bffa0-9768-4e3e-bc0d-8d1f55185420 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:39.812444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1306/2605] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:39 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:39.813780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64054f0-6f16-421c-94a7-0d47a93cf796 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:39.824813 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64054f0-6f16-421c-94a7-0d47a93cf796 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:39.829835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd8e7a-de88-47cf-9921-ed0d9d32d88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:39.830507 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd8e7a-de88-47cf-9921-ed0d9d32d88c admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:39.830823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd8e7a-de88-47cf-9921-ed0d9d32d88c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:39.831103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd8e7a-de88-47cf-9921-ed0d9d32d88c admin admin] PATH_INFO: `/v3/projects/af7781d1a4af4021adb65d16740ef20e/users/01fe05186dbb4477a23b02bf93323907/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:39.847465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd8e7a-de88-47cf-9921-ed0d9d32d88c admin admin] RBAC: Authorizing `identity:create_grant(project_id=af7781d1a4af4021adb65d16740ef20e, user_id=01fe05186dbb4477a23b02bf93323907, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:39.850143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd8e7a-de88-47cf-9921-ed0d9d32d88c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:39.863000 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bffa0-9768-4e3e-bc0d-8d1f55185420 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:39.864835 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-437bffa0-9768-4e3e-bc0d-8d1f55185420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:39.868488 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1300/2606] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:27:39 2022] PUT /identity/v3/projects/af7781d1a4af4021adb65d16740ef20e/users/01fe05186dbb4477a23b02bf93323907/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:27:39.872536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd8e7a-de88-47cf-9921-ed0d9d32d88c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:39.880235 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bffa0-9768-4e3e-bc0d-8d1f55185420 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:39.915898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:39.938179 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd8e7a-de88-47cf-9921-ed0d9d32d88c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:39.943170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:39.943581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00acd2c4-41af-4890-bb55-4cfc79bf678e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:39.943825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:39.944108 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:39.944210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acd2c4-41af-4890-bb55-4cfc79bf678e admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:39.944385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] PATH_INFO: `/v3/projects/f44ad042ba9b4bdc91783fce54774c29` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:39.944487 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acd2c4-41af-4890-bb55-4cfc79bf678e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:39.944781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acd2c4-41af-4890-bb55-4cfc79bf678e admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:39.945485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00acd2c4-41af-4890-bb55-4cfc79bf678e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:39.945596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f44ad042ba9b4bdc91783fce54774c29)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:39.947137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00acd2c4-41af-4890-bb55-4cfc79bf678e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:39.947514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:39.959081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1301/2607] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:39 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:39.963210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00acd2c4-41af-4890-bb55-4cfc79bf678e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:39.973506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00acd2c4-41af-4890-bb55-4cfc79bf678e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:39.979215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91337c-2255-4b0a-be4a-e0a547f5dd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:39.979951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91337c-2255-4b0a-be4a-e0a547f5dd70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:39.980242 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91337c-2255-4b0a-be4a-e0a547f5dd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:39.980520 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91337c-2255-4b0a-be4a-e0a547f5dd70 admin admin] PATH_INFO: `/v3/projects/af7781d1a4af4021adb65d16740ef20e/users/01fe05186dbb4477a23b02bf93323907/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:39.982631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91337c-2255-4b0a-be4a-e0a547f5dd70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af7781d1a4af4021adb65d16740ef20e, user_id=01fe05186dbb4477a23b02bf93323907, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:39.985508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91337c-2255-4b0a-be4a-e0a547f5dd70 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:39.999104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1302/2608] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:27:39 2022] PUT /identity/v3/projects/af7781d1a4af4021adb65d16740ef20e/users/01fe05186dbb4477a23b02bf93323907/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:27:40.004218 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f44ad042ba9b4bdc91783fce54774c29', 'request_id': 'req-a7e638de-390b-4a08-96b1-97b6a2a4c373'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:40.010744 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4db8ca-4885-46d9-ac71-5e1264894809 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:40.011059 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4db8ca-4885-46d9-ac71-5e1264894809 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:40.011318 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4db8ca-4885-46d9-ac71-5e1264894809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:40.012343 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f44ad042ba9b4bdc91783fce54774c29', 'request_id': 'req-a7e638de-390b-4a08-96b1-97b6a2a4c373'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:40.020778 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f44ad042ba9b4bdc91783fce54774c29', 'request_id': 'req-a7e638de-390b-4a08-96b1-97b6a2a4c373'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:40.039208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] The token cache is being invalidate because project f44ad042ba9b4bdc91783fce54774c29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:40.039407 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:40.040128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:40.042363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1307/2609] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:27:39 2022] DELETE /identity/v3/projects/f44ad042ba9b4bdc91783fce54774c29 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:27:40.046251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:40.047869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:40.050016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2f4db8ca-4885-46d9-ac71-5e1264894809 None None] MFA Rules not processed for user `01fe05186dbb4477a23b02bf93323907`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:40.051292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2f4db8ca-4885-46d9-ac71-5e1264894809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:40.092714 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:40.123258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e638de-390b-4a08-96b1-97b6a2a4c373 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:40.136046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1303/2610] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:27:40.148617 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4db8ca-4885-46d9-ac71-5e1264894809 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:40.150524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2f4db8ca-4885-46d9-ac71-5e1264894809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:40.209469 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d51c5-e51a-4f9c-9b55-0d903ab66efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:40.210450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d51c5-e51a-4f9c-9b55-0d903ab66efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:40.211073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d51c5-e51a-4f9c-9b55-0d903ab66efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:40.211557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d51c5-e51a-4f9c-9b55-0d903ab66efb admin admin] PATH_INFO: `/v3/users/2245e963d3c44a5c8aea0d34cd0c4e9e` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:40.217576 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4db8ca-4885-46d9-ac71-5e1264894809 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:40.222158 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729a89a-ace9-4977-ace9-bdab871c78e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:40.222832 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729a89a-ace9-4977-ace9-bdab871c78e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:40.223120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729a89a-ace9-4977-ace9-bdab871c78e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:40.223394 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729a89a-ace9-4977-ace9-bdab871c78e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:40.230267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d51c5-e51a-4f9c-9b55-0d903ab66efb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2245e963d3c44a5c8aea0d34cd0c4e9e)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:40.232832 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d51c5-e51a-4f9c-9b55-0d903ab66efb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:40.234030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729a89a-ace9-4977-ace9-bdab871c78e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:40.237572 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:40.237751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729a89a-ace9-4977-ace9-bdab871c78e8 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:40.250731 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1304/2611] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:27:40.298470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-887d51c5-e51a-4f9c-9b55-0d903ab66efb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2245e963d3c44a5c8aea0d34cd0c4e9e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:40.299809 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-887d51c5-e51a-4f9c-9b55-0d903ab66efb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2245e963d3c44a5c8aea0d34cd0c4e9e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:40.307257 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-887d51c5-e51a-4f9c-9b55-0d903ab66efb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2245e963d3c44a5c8aea0d34cd0c4e9e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:40.314695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-887d51c5-e51a-4f9c-9b55-0d903ab66efb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2245e963d3c44a5c8aea0d34cd0c4e9e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:40.331221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-887d51c5-e51a-4f9c-9b55-0d903ab66efb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2245e963d3c44a5c8aea0d34cd0c4e9e'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:40.347655 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1308/2612] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:27:40 2022] DELETE /identity/v3/users/2245e963d3c44a5c8aea0d34cd0c4e9e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:27:40.368834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729a89a-ace9-4977-ace9-bdab871c78e8 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:40.378347 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729a89a-ace9-4977-ace9-bdab871c78e8 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:40.440263 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1c058-03c2-495a-9821-1fbacdcbdc80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:40.440890 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1c058-03c2-495a-9821-1fbacdcbdc80 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:40.441173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1c058-03c2-495a-9821-1fbacdcbdc80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:40.441447 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1c058-03c2-495a-9821-1fbacdcbdc80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:40.443271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-22b1c058-03c2-495a-9821-1fbacdcbdc80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:40.507424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1c058-03c2-495a-9821-1fbacdcbdc80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:40.508885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1c058-03c2-495a-9821-1fbacdcbdc80 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:40.519334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1305/2613] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:40.896833 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d51c5-e51a-4f9c-9b55-0d903ab66efb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:40.898503 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-887d51c5-e51a-4f9c-9b55-0d903ab66efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:40.975394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d51c5-e51a-4f9c-9b55-0d903ab66efb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:40.981772 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:40.982426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:40.982746 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:40.983027 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] PATH_INFO: `/v3/projects/2371e97d1d20446db4e42d117ff84937` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:40.994447 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2371e97d1d20446db4e42d117ff84937)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:40.996512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:41.046219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2371e97d1d20446db4e42d117ff84937', 'request_id': 'req-54f80518-5373-443b-840b-c091f3d5161f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:41.054151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2371e97d1d20446db4e42d117ff84937', 'request_id': 'req-54f80518-5373-443b-840b-c091f3d5161f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:41.060224 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2371e97d1d20446db4e42d117ff84937', 'request_id': 'req-54f80518-5373-443b-840b-c091f3d5161f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:41.073311 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] The token cache is being invalidate because project 2371e97d1d20446db4e42d117ff84937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:41.073632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:41.074246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:41.076062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1309/2614] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:27:40 2022] DELETE /identity/v3/projects/2371e97d1d20446db4e42d117ff84937 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:45.170247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1c058-03c2-495a-9821-1fbacdcbdc80 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:45.171689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-22b1c058-03c2-495a-9821-1fbacdcbdc80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:45.210114 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:45.238528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1c058-03c2-495a-9821-1fbacdcbdc80 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:45.291774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3f0f4-5ff2-46dd-8406-853c7e70c2d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:45.292666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3f0f4-5ff2-46dd-8406-853c7e70c2d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:45.292955 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3f0f4-5ff2-46dd-8406-853c7e70c2d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:45.293235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3f0f4-5ff2-46dd-8406-853c7e70c2d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:45.295129 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4fc3f0f4-5ff2-46dd-8406-853c7e70c2d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:45.365024 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:45.366639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:45.368623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3f0f4-5ff2-46dd-8406-853c7e70c2d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:45.370078 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3f0f4-5ff2-46dd-8406-853c7e70c2d7 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:45.374883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:45.380313 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1306/2615] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:45.380457 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f80518-5373-443b-840b-c091f3d5161f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:45.386487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a34445-00a6-4944-8814-55901c5e25bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:45.387307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a34445-00a6-4944-8814-55901c5e25bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:45.387607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a34445-00a6-4944-8814-55901c5e25bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:45.387898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a34445-00a6-4944-8814-55901c5e25bb admin admin] PATH_INFO: `/v3/users/6c570d965ffa49c69919e9cc08c71b87` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:45.389127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a34445-00a6-4944-8814-55901c5e25bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c570d965ffa49c69919e9cc08c71b87)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:45.391194 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a34445-00a6-4944-8814-55901c5e25bb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:45.441328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-23a34445-00a6-4944-8814-55901c5e25bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c570d965ffa49c69919e9cc08c71b87'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:45.442069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-23a34445-00a6-4944-8814-55901c5e25bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c570d965ffa49c69919e9cc08c71b87'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:45.449542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-23a34445-00a6-4944-8814-55901c5e25bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c570d965ffa49c69919e9cc08c71b87'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:45.450276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-23a34445-00a6-4944-8814-55901c5e25bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c570d965ffa49c69919e9cc08c71b87'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:45.463284 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-23a34445-00a6-4944-8814-55901c5e25bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c570d965ffa49c69919e9cc08c71b87'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:45.477842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1310/2616] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:27:45 2022] DELETE /identity/v3/users/6c570d965ffa49c69919e9cc08c71b87 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:27:45.482353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619315df-5751-4db2-807a-add167072d87 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:45.482630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619315df-5751-4db2-807a-add167072d87 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:45.482943 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619315df-5751-4db2-807a-add167072d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:45.502751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-619315df-5751-4db2-807a-add167072d87 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:45.503979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-619315df-5751-4db2-807a-add167072d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:45.580814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1307/2617] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:45.591271 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a34445-00a6-4944-8814-55901c5e25bb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:45.592709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-23a34445-00a6-4944-8814-55901c5e25bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:45.652391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a34445-00a6-4944-8814-55901c5e25bb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:45.656887 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a47b8-f48d-43dc-a850-285c32acb2fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:45.657613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a47b8-f48d-43dc-a850-285c32acb2fc service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:45.657898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a47b8-f48d-43dc-a850-285c32acb2fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:45.658179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a47b8-f48d-43dc-a850-285c32acb2fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:45.668189 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a47b8-f48d-43dc-a850-285c32acb2fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:45.669633 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a47b8-f48d-43dc-a850-285c32acb2fc service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:45.680314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1311/2618] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:45.958477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619315df-5751-4db2-807a-add167072d87 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:45.959951 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-619315df-5751-4db2-807a-add167072d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:45.982531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619315df-5751-4db2-807a-add167072d87 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:45.990725 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:45.991380 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:45.991662 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:45.991938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] PATH_INFO: `/v3/projects/cf6a4011eeac4641b96fba21ce06d55a` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:45.993121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf6a4011eeac4641b96fba21ce06d55a)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:45.995079 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:46.031786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:46.047516 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf6a4011eeac4641b96fba21ce06d55a', 'request_id': 'req-d7d0f219-1f8b-4230-ac04-40de80060243'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.054876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf6a4011eeac4641b96fba21ce06d55a', 'request_id': 'req-d7d0f219-1f8b-4230-ac04-40de80060243'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.063847 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf6a4011eeac4641b96fba21ce06d55a', 'request_id': 'req-d7d0f219-1f8b-4230-ac04-40de80060243'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.076246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] The token cache is being invalidate because project cf6a4011eeac4641b96fba21ce06d55a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:46.076659 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.077309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.078867 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1308/2619] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:27:45 2022] DELETE /identity/v3/projects/cf6a4011eeac4641b96fba21ce06d55a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:46.083525 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a47b8-f48d-43dc-a850-285c32acb2fc service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:46.093820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a47b8-f48d-43dc-a850-285c32acb2fc service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:46.155742 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d226f880-b9ca-4f68-a4fb-42ed54806922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:46.156485 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226f880-b9ca-4f68-a4fb-42ed54806922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:46.156839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226f880-b9ca-4f68-a4fb-42ed54806922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:46.157191 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226f880-b9ca-4f68-a4fb-42ed54806922 admin admin] PATH_INFO: `/v3/users/01fe05186dbb4477a23b02bf93323907` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:46.158473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226f880-b9ca-4f68-a4fb-42ed54806922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01fe05186dbb4477a23b02bf93323907)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:46.160538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d226f880-b9ca-4f68-a4fb-42ed54806922 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:46.182791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:46.210623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d226f880-b9ca-4f68-a4fb-42ed54806922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01fe05186dbb4477a23b02bf93323907'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.211351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d226f880-b9ca-4f68-a4fb-42ed54806922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01fe05186dbb4477a23b02bf93323907'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.218748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d226f880-b9ca-4f68-a4fb-42ed54806922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01fe05186dbb4477a23b02bf93323907'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.219415 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d226f880-b9ca-4f68-a4fb-42ed54806922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01fe05186dbb4477a23b02bf93323907'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.231598 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d226f880-b9ca-4f68-a4fb-42ed54806922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01fe05186dbb4477a23b02bf93323907'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.246183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1312/2620] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:27:46 2022] DELETE /identity/v3/users/01fe05186dbb4477a23b02bf93323907 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:46.683226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:46.684909 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:46.698516 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d0f219-1f8b-4230-ac04-40de80060243 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:46.766160 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91508f-6822-4c2b-af49-8fce25c305e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:46.766850 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91508f-6822-4c2b-af49-8fce25c305e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:46.767136 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91508f-6822-4c2b-af49-8fce25c305e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:46.767416 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91508f-6822-4c2b-af49-8fce25c305e5 admin admin] PATH_INFO: `/v3/projects/af7781d1a4af4021adb65d16740ef20e` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:46.769348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91508f-6822-4c2b-af49-8fce25c305e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af7781d1a4af4021adb65d16740ef20e)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:46.771327 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91508f-6822-4c2b-af49-8fce25c305e5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:46.824610 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0c91508f-6822-4c2b-af49-8fce25c305e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af7781d1a4af4021adb65d16740ef20e', 'request_id': 'req-0c91508f-6822-4c2b-af49-8fce25c305e5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.832471 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0c91508f-6822-4c2b-af49-8fce25c305e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af7781d1a4af4021adb65d16740ef20e', 'request_id': 'req-0c91508f-6822-4c2b-af49-8fce25c305e5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.841228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0c91508f-6822-4c2b-af49-8fce25c305e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af7781d1a4af4021adb65d16740ef20e', 'request_id': 'req-0c91508f-6822-4c2b-af49-8fce25c305e5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.852837 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0c91508f-6822-4c2b-af49-8fce25c305e5 admin admin] The token cache is being invalidate because project af7781d1a4af4021adb65d16740ef20e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:46.853144 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0c91508f-6822-4c2b-af49-8fce25c305e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.853876 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-0c91508f-6822-4c2b-af49-8fce25c305e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:46.855448 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1309/2621] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:27:46 2022] DELETE /identity/v3/projects/af7781d1a4af4021adb65d16740ef20e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:27:46.861956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8053194-71bd-42b1-bebc-30a239a34b2f None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:46.862438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8053194-71bd-42b1-bebc-30a239a34b2f None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:46.862549 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8053194-71bd-42b1-bebc-30a239a34b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:46.891918 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d8053194-71bd-42b1-bebc-30a239a34b2f None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:46.893795 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d8053194-71bd-42b1-bebc-30a239a34b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:46.975907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1313/2622] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:46.985136 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1305cd-dd85-470b-9670-b5935c8968e0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:46.985627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1305cd-dd85-470b-9670-b5935c8968e0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:46.985907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1305cd-dd85-470b-9670-b5935c8968e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:47.008348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-fa1305cd-dd85-470b-9670-b5935c8968e0 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:47.009583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-fa1305cd-dd85-470b-9670-b5935c8968e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:47.019094 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1310/2623] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:27:47.023604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8053194-71bd-42b1-bebc-30a239a34b2f None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:47.037444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8053194-71bd-42b1-bebc-30a239a34b2f None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:47.042152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5937b54-0c30-4534-b683-d5f461408a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:47.042999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5937b54-0c30-4534-b683-d5f461408a47 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:47.043385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5937b54-0c30-4534-b683-d5f461408a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:47.043803 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5937b54-0c30-4534-b683-d5f461408a47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:47.044620 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5937b54-0c30-4534-b683-d5f461408a47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:27:47.045024 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5937b54-0c30-4534-b683-d5f461408a47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:47.046935 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5937b54-0c30-4534-b683-d5f461408a47 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:47.062274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1314/2624] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:27:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 15:27:47.067289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1305cd-dd85-470b-9670-b5935c8968e0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:47.083264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1305cd-dd85-470b-9670-b5935c8968e0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:47.088154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b89af3-6171-4200-93f8-2cbe74432b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:47.089061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b89af3-6171-4200-93f8-2cbe74432b5e admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:47.089714 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b89af3-6171-4200-93f8-2cbe74432b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:47.090017 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b89af3-6171-4200-93f8-2cbe74432b5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:47.090883 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b89af3-6171-4200-93f8-2cbe74432b5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:47.092815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b89af3-6171-4200-93f8-2cbe74432b5e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:47.129043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1311/2625] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:27:47 2022] POST /identity/v3/projects => generated 398 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:27:47.133585 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5937b54-0c30-4534-b683-d5f461408a47 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:47.153439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5937b54-0c30-4534-b683-d5f461408a47 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:47.230775 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c82dd-21af-42c1-a06e-998afff0e712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:47.231425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c82dd-21af-42c1-a06e-998afff0e712 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:47.231786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c82dd-21af-42c1-a06e-998afff0e712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:47.232147 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c82dd-21af-42c1-a06e-998afff0e712 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:47.232901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c82dd-21af-42c1-a06e-998afff0e712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:47.234941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c82dd-21af-42c1-a06e-998afff0e712 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:47.268524 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1315/2626] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:27:47 2022] POST /identity/v3/users => generated 300 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:27:47.272727 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b89af3-6171-4200-93f8-2cbe74432b5e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:47.285511 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b89af3-6171-4200-93f8-2cbe74432b5e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:47.291109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21015831-db97-4bd6-80e4-be747e7b2d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:47.291846 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21015831-db97-4bd6-80e4-be747e7b2d54 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:47.292223 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21015831-db97-4bd6-80e4-be747e7b2d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:47.292589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21015831-db97-4bd6-80e4-be747e7b2d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:47.293385 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21015831-db97-4bd6-80e4-be747e7b2d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:47.295107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21015831-db97-4bd6-80e4-be747e7b2d54 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:47.307159 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1312/2627] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:47 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:47.311183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c82dd-21af-42c1-a06e-998afff0e712 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:47.322769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c82dd-21af-42c1-a06e-998afff0e712 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:47.327202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eabf39-6c25-4c10-bbca-961f21e40895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:47.327926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eabf39-6c25-4c10-bbca-961f21e40895 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:47.328315 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eabf39-6c25-4c10-bbca-961f21e40895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:47.328711 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eabf39-6c25-4c10-bbca-961f21e40895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:47.329480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eabf39-6c25-4c10-bbca-961f21e40895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:47.331208 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eabf39-6c25-4c10-bbca-961f21e40895 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:47.342780 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1316/2628] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:47 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:47.347621 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21015831-db97-4bd6-80e4-be747e7b2d54 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:47.358161 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21015831-db97-4bd6-80e4-be747e7b2d54 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:47.363410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfa920-a0a8-4c61-98a6-4a7a5bb9ff49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:47.364077 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfa920-a0a8-4c61-98a6-4a7a5bb9ff49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:47.364355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfa920-a0a8-4c61-98a6-4a7a5bb9ff49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:47.364654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfa920-a0a8-4c61-98a6-4a7a5bb9ff49 admin admin] PATH_INFO: `/v3/projects/9ff52aead0ac4baeb5dd4bbf4852650f/users/732cb8f3d9014c089dcb82600733018b/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:47.368916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:47.386521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfa920-a0a8-4c61-98a6-4a7a5bb9ff49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff52aead0ac4baeb5dd4bbf4852650f, user_id=732cb8f3d9014c089dcb82600733018b, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:47.390210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfa920-a0a8-4c61-98a6-4a7a5bb9ff49 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:47.406934 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1313/2629] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:27:47 2022] PUT /identity/v3/projects/9ff52aead0ac4baeb5dd4bbf4852650f/users/732cb8f3d9014c089dcb82600733018b/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:47.417932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38db7d-6202-4955-a107-6e51fa32af60 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:47.418355 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38db7d-6202-4955-a107-6e51fa32af60 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:47.418797 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38db7d-6202-4955-a107-6e51fa32af60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:47.426975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:47.452924 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-de38db7d-6202-4955-a107-6e51fa32af60 None None] MFA Rules not processed for user `732cb8f3d9014c089dcb82600733018b`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:47.454335 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-de38db7d-6202-4955-a107-6e51fa32af60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:47.545664 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1317/2630] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:47 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:47.557669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfa920-a0a8-4c61-98a6-4a7a5bb9ff49 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:47.559485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0adfa920-a0a8-4c61-98a6-4a7a5bb9ff49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:47.617716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfa920-a0a8-4c61-98a6-4a7a5bb9ff49 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:47.621557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7967f8-c917-491e-9196-e311f9360f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:47.622274 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7967f8-c917-491e-9196-e311f9360f3e service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:47.622565 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7967f8-c917-491e-9196-e311f9360f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:47.622872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7967f8-c917-491e-9196-e311f9360f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:47.631759 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7967f8-c917-491e-9196-e311f9360f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:47.633202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7967f8-c917-491e-9196-e311f9360f3e service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:47.642535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1314/2631] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:47 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:48.540154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38db7d-6202-4955-a107-6e51fa32af60 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:48.544561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-de38db7d-6202-4955-a107-6e51fa32af60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:48.623934 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38db7d-6202-4955-a107-6e51fa32af60 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:48.629214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc6c54-e361-46be-93f8-4a5ddf42ef2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:48.635265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc6c54-e361-46be-93f8-4a5ddf42ef2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:48.635265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc6c54-e361-46be-93f8-4a5ddf42ef2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:48.635265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc6c54-e361-46be-93f8-4a5ddf42ef2a admin admin] PATH_INFO: `/v3/users/424ccf230e16423a8dd32097d90c4995` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:48.635265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc6c54-e361-46be-93f8-4a5ddf42ef2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=424ccf230e16423a8dd32097d90c4995)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:48.635265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc6c54-e361-46be-93f8-4a5ddf42ef2a admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:48.685425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-41bc6c54-e361-46be-93f8-4a5ddf42ef2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '424ccf230e16423a8dd32097d90c4995'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:48.686082 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-41bc6c54-e361-46be-93f8-4a5ddf42ef2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '424ccf230e16423a8dd32097d90c4995'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:48.693637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-41bc6c54-e361-46be-93f8-4a5ddf42ef2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '424ccf230e16423a8dd32097d90c4995'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:48.694274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-41bc6c54-e361-46be-93f8-4a5ddf42ef2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '424ccf230e16423a8dd32097d90c4995'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:48.706463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-41bc6c54-e361-46be-93f8-4a5ddf42ef2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '424ccf230e16423a8dd32097d90c4995'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:48.719348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1318/2632] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:27:48 2022] DELETE /identity/v3/users/424ccf230e16423a8dd32097d90c4995 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:48.900881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7967f8-c917-491e-9196-e311f9360f3e service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:48.914372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7967f8-c917-491e-9196-e311f9360f3e service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:48.981236 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde0149-a3f0-4286-b52f-6d55c128823b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:48.981873 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde0149-a3f0-4286-b52f-6d55c128823b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:48.982154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde0149-a3f0-4286-b52f-6d55c128823b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:48.982432 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde0149-a3f0-4286-b52f-6d55c128823b admin admin] PATH_INFO: `/v3/projects/2bb9ac7a058f47f39a01b9bd92c7f4a2` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:48.983720 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde0149-a3f0-4286-b52f-6d55c128823b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bb9ac7a058f47f39a01b9bd92c7f4a2)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:48.985702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccde0149-a3f0-4286-b52f-6d55c128823b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:49.037334 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ccde0149-a3f0-4286-b52f-6d55c128823b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bb9ac7a058f47f39a01b9bd92c7f4a2', 'request_id': 'req-ccde0149-a3f0-4286-b52f-6d55c128823b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:49.043630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ccde0149-a3f0-4286-b52f-6d55c128823b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bb9ac7a058f47f39a01b9bd92c7f4a2', 'request_id': 'req-ccde0149-a3f0-4286-b52f-6d55c128823b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:49.052156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ccde0149-a3f0-4286-b52f-6d55c128823b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bb9ac7a058f47f39a01b9bd92c7f4a2', 'request_id': 'req-ccde0149-a3f0-4286-b52f-6d55c128823b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:49.062886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ccde0149-a3f0-4286-b52f-6d55c128823b admin admin] The token cache is being invalidate because project 2bb9ac7a058f47f39a01b9bd92c7f4a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:49.063182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ccde0149-a3f0-4286-b52f-6d55c128823b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:49.063821 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ccde0149-a3f0-4286-b52f-6d55c128823b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:49.065388 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1315/2633] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:27:48 2022] DELETE /identity/v3/projects/2bb9ac7a058f47f39a01b9bd92c7f4a2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:49.070468 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e3d6e-ca5a-4555-8be8-2100c74cf5c1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:49.070791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e3d6e-ca5a-4555-8be8-2100c74cf5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:49.071081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e3d6e-ca5a-4555-8be8-2100c74cf5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:49.093176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6f7e3d6e-ca5a-4555-8be8-2100c74cf5c1 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:49.094406 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6f7e3d6e-ca5a-4555-8be8-2100c74cf5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:49.100919 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:49.165742 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1319/2634] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:49.178905 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e25298-1f05-4f67-b553-9f916ae7b101 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:49.178905 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e25298-1f05-4f67-b553-9f916ae7b101 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:49.178905 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e25298-1f05-4f67-b553-9f916ae7b101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:49.204584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a0e25298-1f05-4f67-b553-9f916ae7b101 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:49.206770 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a0e25298-1f05-4f67-b553-9f916ae7b101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:49.215019 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1316/2635] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:49.219915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e3d6e-ca5a-4555-8be8-2100c74cf5c1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:49.231124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e3d6e-ca5a-4555-8be8-2100c74cf5c1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:49.235962 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78349ae1-f87c-42db-804e-2b880c0862aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:49.236576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78349ae1-f87c-42db-804e-2b880c0862aa admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:49.236861 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78349ae1-f87c-42db-804e-2b880c0862aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:49.237139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78349ae1-f87c-42db-804e-2b880c0862aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:49.237837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78349ae1-f87c-42db-804e-2b880c0862aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:27:49.238158 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78349ae1-f87c-42db-804e-2b880c0862aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:49.240063 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78349ae1-f87c-42db-804e-2b880c0862aa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:49.251841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1320/2636] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:27:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:27:49.256393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e25298-1f05-4f67-b553-9f916ae7b101 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:49.266588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e25298-1f05-4f67-b553-9f916ae7b101 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:49.271862 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc9288-58d2-426a-8aeb-50e7845b6446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:49.272487 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc9288-58d2-426a-8aeb-50e7845b6446 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:49.272854 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc9288-58d2-426a-8aeb-50e7845b6446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:49.273177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bc9288-58d2-426a-8aeb-50e7845b6446 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:49.273960 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc9288-58d2-426a-8aeb-50e7845b6446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:49.276083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bc9288-58d2-426a-8aeb-50e7845b6446 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:49.288285 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:49.308289 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1317/2637] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:27:49 2022] POST /identity/v3/projects => generated 364 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:27:49.312653 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78349ae1-f87c-42db-804e-2b880c0862aa admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:49.322706 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78349ae1-f87c-42db-804e-2b880c0862aa admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:49.389439 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50b9c9-816a-46c8-9fd9-b45a8acd4849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:49.390073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50b9c9-816a-46c8-9fd9-b45a8acd4849 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:49.390441 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50b9c9-816a-46c8-9fd9-b45a8acd4849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:49.390750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50b9c9-816a-46c8-9fd9-b45a8acd4849 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:49.391423 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50b9c9-816a-46c8-9fd9-b45a8acd4849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:49.393878 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50b9c9-816a-46c8-9fd9-b45a8acd4849 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:49.416059 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1321/2638] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:27:49 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:27:49.420168 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc9288-58d2-426a-8aeb-50e7845b6446 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:49.431217 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bc9288-58d2-426a-8aeb-50e7845b6446 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:49.435761 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e1693-5d5e-4bb8-8109-9316b279d8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:49.436357 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e1693-5d5e-4bb8-8109-9316b279d8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:49.436668 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e1693-5d5e-4bb8-8109-9316b279d8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:49.436939 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e1693-5d5e-4bb8-8109-9316b279d8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:49.437622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e1693-5d5e-4bb8-8109-9316b279d8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:49.439249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e1693-5d5e-4bb8-8109-9316b279d8f5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:49.450121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1318/2639] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:49 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:49.454894 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50b9c9-816a-46c8-9fd9-b45a8acd4849 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:49.465692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50b9c9-816a-46c8-9fd9-b45a8acd4849 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:49.470314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ebf84-515e-4e51-b37f-2b7dbc97ce6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:49.471610 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ebf84-515e-4e51-b37f-2b7dbc97ce6f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:49.471892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ebf84-515e-4e51-b37f-2b7dbc97ce6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:49.472168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ebf84-515e-4e51-b37f-2b7dbc97ce6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:49.472858 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ebf84-515e-4e51-b37f-2b7dbc97ce6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:49.474477 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ebf84-515e-4e51-b37f-2b7dbc97ce6f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:49.485074 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1322/2640] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:27:49 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:27:49.489036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e1693-5d5e-4bb8-8109-9316b279d8f5 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:49.498843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e1693-5d5e-4bb8-8109-9316b279d8f5 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:49.503441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625152b3-a881-48e0-9990-f45f723d9f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:49.504092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625152b3-a881-48e0-9990-f45f723d9f8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:49.504376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625152b3-a881-48e0-9990-f45f723d9f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:49.504650 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625152b3-a881-48e0-9990-f45f723d9f8c admin admin] PATH_INFO: `/v3/projects/5093da36d95e4b159210982225723bb7/users/3918efb3cb1f4171b3715d48328eebe7/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:49.519561 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625152b3-a881-48e0-9990-f45f723d9f8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5093da36d95e4b159210982225723bb7, user_id=3918efb3cb1f4171b3715d48328eebe7, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:49.522246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625152b3-a881-48e0-9990-f45f723d9f8c admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:49.535615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1319/2641] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:27:49 2022] PUT /identity/v3/projects/5093da36d95e4b159210982225723bb7/users/3918efb3cb1f4171b3715d48328eebe7/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:49.547412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a1264-f3c2-4bba-ba08-32542432ec8b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:49.547794 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a1264-f3c2-4bba-ba08-32542432ec8b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:49.547913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a1264-f3c2-4bba-ba08-32542432ec8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:49.575738 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-aa8a1264-f3c2-4bba-ba08-32542432ec8b None None] MFA Rules not processed for user `3918efb3cb1f4171b3715d48328eebe7`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:49.576961 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-aa8a1264-f3c2-4bba-ba08-32542432ec8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:49.641228 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1323/2642] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:49.653295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625152b3-a881-48e0-9990-f45f723d9f8c admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:49.655130 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-625152b3-a881-48e0-9990-f45f723d9f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:49.713016 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625152b3-a881-48e0-9990-f45f723d9f8c admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:49.716700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b577a31-d659-4ec0-a770-7cbdeefe1c57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:49.717319 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b577a31-d659-4ec0-a770-7cbdeefe1c57 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:49.717596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b577a31-d659-4ec0-a770-7cbdeefe1c57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:49.717872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b577a31-d659-4ec0-a770-7cbdeefe1c57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:49.726080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b577a31-d659-4ec0-a770-7cbdeefe1c57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:49.727546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b577a31-d659-4ec0-a770-7cbdeefe1c57 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:49.736338 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1320/2643] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:56.833411 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a1264-f3c2-4bba-ba08-32542432ec8b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:56.834910 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-aa8a1264-f3c2-4bba-ba08-32542432ec8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:56.844348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:56.900883 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a1264-f3c2-4bba-ba08-32542432ec8b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:56.905306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb66774-56d7-4a5d-adf5-d5845f8bf31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:56.905940 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb66774-56d7-4a5d-adf5-d5845f8bf31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:56.906222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb66774-56d7-4a5d-adf5-d5845f8bf31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:56.906494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb66774-56d7-4a5d-adf5-d5845f8bf31f admin admin] PATH_INFO: `/v3/users/732cb8f3d9014c089dcb82600733018b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:56.907690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb66774-56d7-4a5d-adf5-d5845f8bf31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=732cb8f3d9014c089dcb82600733018b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:56.909588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb66774-56d7-4a5d-adf5-d5845f8bf31f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:56.953380 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbb66774-56d7-4a5d-adf5-d5845f8bf31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '732cb8f3d9014c089dcb82600733018b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:56.954031 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbb66774-56d7-4a5d-adf5-d5845f8bf31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '732cb8f3d9014c089dcb82600733018b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:56.962071 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbb66774-56d7-4a5d-adf5-d5845f8bf31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '732cb8f3d9014c089dcb82600733018b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:56.962814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbb66774-56d7-4a5d-adf5-d5845f8bf31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '732cb8f3d9014c089dcb82600733018b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:56.975371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-dbb66774-56d7-4a5d-adf5-d5845f8bf31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '732cb8f3d9014c089dcb82600733018b'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:56.988839 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1324/2644] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:27:56 2022] DELETE /identity/v3/users/732cb8f3d9014c089dcb82600733018b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:27:56.993453 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fad1fc-b483-4d46-ad99-86720bb0c210 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:56.993912 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fad1fc-b483-4d46-ad99-86720bb0c210 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:56.994046 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fad1fc-b483-4d46-ad99-86720bb0c210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:57.013491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-13fad1fc-b483-4d46-ad99-86720bb0c210 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:27:57.014866 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-13fad1fc-b483-4d46-ad99-86720bb0c210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:57.087091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1321/2645] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:27:57.096673 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb66774-56d7-4a5d-adf5-d5845f8bf31f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:57.098176 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dbb66774-56d7-4a5d-adf5-d5845f8bf31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:57.157488 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb66774-56d7-4a5d-adf5-d5845f8bf31f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:57.161333 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec7de4-2552-42f6-8d8d-ac065a19f910 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:57.161954 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec7de4-2552-42f6-8d8d-ac065a19f910 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:57.162237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec7de4-2552-42f6-8d8d-ac065a19f910 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:57.162513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec7de4-2552-42f6-8d8d-ac065a19f910 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:57.172155 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec7de4-2552-42f6-8d8d-ac065a19f910 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:57.173582 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ec7de4-2552-42f6-8d8d-ac065a19f910 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:57.183476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1325/2646] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:27:57.476030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fad1fc-b483-4d46-ad99-86720bb0c210 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:27:57.478097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-13fad1fc-b483-4d46-ad99-86720bb0c210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:27:57.501428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fad1fc-b483-4d46-ad99-86720bb0c210 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:27:57.507996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607f083-de58-474a-bcf0-0c981c26eb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:27:57.508649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607f083-de58-474a-bcf0-0c981c26eb3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:27:57.508934 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607f083-de58-474a-bcf0-0c981c26eb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:27:57.509216 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607f083-de58-474a-bcf0-0c981c26eb3f admin admin] PATH_INFO: `/v3/projects/9ff52aead0ac4baeb5dd4bbf4852650f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:27:57.510428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607f083-de58-474a-bcf0-0c981c26eb3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ff52aead0ac4baeb5dd4bbf4852650f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:27:57.512371 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607f083-de58-474a-bcf0-0c981c26eb3f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:27:57.566171 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f607f083-de58-474a-bcf0-0c981c26eb3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ff52aead0ac4baeb5dd4bbf4852650f', 'request_id': 'req-f607f083-de58-474a-bcf0-0c981c26eb3f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:57.568639 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:27:57.573974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f607f083-de58-474a-bcf0-0c981c26eb3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ff52aead0ac4baeb5dd4bbf4852650f', 'request_id': 'req-f607f083-de58-474a-bcf0-0c981c26eb3f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:57.583948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f607f083-de58-474a-bcf0-0c981c26eb3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ff52aead0ac4baeb5dd4bbf4852650f', 'request_id': 'req-f607f083-de58-474a-bcf0-0c981c26eb3f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:57.595146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f607f083-de58-474a-bcf0-0c981c26eb3f admin admin] The token cache is being invalidate because project 9ff52aead0ac4baeb5dd4bbf4852650f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:27:57.595508 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f607f083-de58-474a-bcf0-0c981c26eb3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:57.596196 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f607f083-de58-474a-bcf0-0c981c26eb3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:27:57.597815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1322/2647] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:27:57 2022] DELETE /identity/v3/projects/9ff52aead0ac4baeb5dd4bbf4852650f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:01.311588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec7de4-2552-42f6-8d8d-ac065a19f910 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:01.316811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-a1ec7de4-2552-42f6-8d8d-ac065a19f910 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:01.329956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ec7de4-2552-42f6-8d8d-ac065a19f910 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:01.359426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:01.393868 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee932ac0-9a1c-4d9d-b89e-35cb07c7e12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:01.394553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee932ac0-9a1c-4d9d-b89e-35cb07c7e12d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:01.395316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee932ac0-9a1c-4d9d-b89e-35cb07c7e12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:01.395592 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee932ac0-9a1c-4d9d-b89e-35cb07c7e12d admin admin] PATH_INFO: `/v3/users/3918efb3cb1f4171b3715d48328eebe7` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:01.396816 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee932ac0-9a1c-4d9d-b89e-35cb07c7e12d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3918efb3cb1f4171b3715d48328eebe7)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:01.398763 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee932ac0-9a1c-4d9d-b89e-35cb07c7e12d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:01.447152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ee932ac0-9a1c-4d9d-b89e-35cb07c7e12d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3918efb3cb1f4171b3715d48328eebe7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:01.447896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ee932ac0-9a1c-4d9d-b89e-35cb07c7e12d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3918efb3cb1f4171b3715d48328eebe7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:01.455304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ee932ac0-9a1c-4d9d-b89e-35cb07c7e12d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3918efb3cb1f4171b3715d48328eebe7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:01.455973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ee932ac0-9a1c-4d9d-b89e-35cb07c7e12d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3918efb3cb1f4171b3715d48328eebe7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:01.469805 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ee932ac0-9a1c-4d9d-b89e-35cb07c7e12d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3918efb3cb1f4171b3715d48328eebe7'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:01.483030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1326/2648] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:28:01 2022] DELETE /identity/v3/users/3918efb3cb1f4171b3715d48328eebe7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:01.487594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf95154-32af-4bb5-9346-21555a8a05ab None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:01.488113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf95154-32af-4bb5-9346-21555a8a05ab None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:01.488311 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf95154-32af-4bb5-9346-21555a8a05ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:01.509177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-dcf95154-32af-4bb5-9346-21555a8a05ab None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:01.510405 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-dcf95154-32af-4bb5-9346-21555a8a05ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:01.576203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1323/2649] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:01.587040 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee932ac0-9a1c-4d9d-b89e-35cb07c7e12d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:01.588946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ee932ac0-9a1c-4d9d-b89e-35cb07c7e12d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:01.649825 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee932ac0-9a1c-4d9d-b89e-35cb07c7e12d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:01.653547 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4953e-9b1d-4c01-bd1a-a9f8a7176ef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:01.654184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4953e-9b1d-4c01-bd1a-a9f8a7176ef7 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:01.654466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4953e-9b1d-4c01-bd1a-a9f8a7176ef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:01.654775 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d4953e-9b1d-4c01-bd1a-a9f8a7176ef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:01.664104 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4953e-9b1d-4c01-bd1a-a9f8a7176ef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:01.665554 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d4953e-9b1d-4c01-bd1a-a9f8a7176ef7 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:01.675683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1327/2650] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:01.982115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf95154-32af-4bb5-9346-21555a8a05ab None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:01.984117 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-dcf95154-32af-4bb5-9346-21555a8a05ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:01.995243 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4953e-9b1d-4c01-bd1a-a9f8a7176ef7 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:01.996773 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-89d4953e-9b1d-4c01-bd1a-a9f8a7176ef7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:02.003666 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf95154-32af-4bb5-9346-21555a8a05ab None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:02.010075 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57ccbd-1c54-4539-abfc-b79863c986d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:02.011093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57ccbd-1c54-4539-abfc-b79863c986d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:02.011377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57ccbd-1c54-4539-abfc-b79863c986d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:02.011711 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57ccbd-1c54-4539-abfc-b79863c986d0 admin admin] PATH_INFO: `/v3/projects/5093da36d95e4b159210982225723bb7` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:02.013124 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57ccbd-1c54-4539-abfc-b79863c986d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5093da36d95e4b159210982225723bb7)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:02.016163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57ccbd-1c54-4539-abfc-b79863c986d0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:02.076812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:02.089349 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c57ccbd-1c54-4539-abfc-b79863c986d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5093da36d95e4b159210982225723bb7', 'request_id': 'req-1c57ccbd-1c54-4539-abfc-b79863c986d0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:02.097473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c57ccbd-1c54-4539-abfc-b79863c986d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5093da36d95e4b159210982225723bb7', 'request_id': 'req-1c57ccbd-1c54-4539-abfc-b79863c986d0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:02.104077 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d4953e-9b1d-4c01-bd1a-a9f8a7176ef7 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:02.107198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c57ccbd-1c54-4539-abfc-b79863c986d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5093da36d95e4b159210982225723bb7', 'request_id': 'req-1c57ccbd-1c54-4539-abfc-b79863c986d0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:02.119965 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c57ccbd-1c54-4539-abfc-b79863c986d0 admin admin] The token cache is being invalidate because project 5093da36d95e4b159210982225723bb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:28:02.120263 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c57ccbd-1c54-4539-abfc-b79863c986d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:02.120892 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c57ccbd-1c54-4539-abfc-b79863c986d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:02.125777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1324/2651] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:28:01 2022] DELETE /identity/v3/projects/5093da36d95e4b159210982225723bb7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:02.127342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc72bb7-bc36-479e-9c28-338167fb3365 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:02.127702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc72bb7-bc36-479e-9c28-338167fb3365 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:02.128035 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc72bb7-bc36-479e-9c28-338167fb3365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:02.148279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1bc72bb7-bc36-479e-9c28-338167fb3365 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:02.149542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1bc72bb7-bc36-479e-9c28-338167fb3365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:02.175958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37457998-64f4-4b7d-b6a6-ca198f40af6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:02.176673 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37457998-64f4-4b7d-b6a6-ca198f40af6a service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:02.177030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37457998-64f4-4b7d-b6a6-ca198f40af6a service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:02.177560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37457998-64f4-4b7d-b6a6-ca198f40af6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:02.179274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-37457998-64f4-4b7d-b6a6-ca198f40af6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:02.191207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37457998-64f4-4b7d-b6a6-ca198f40af6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:02.192890 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37457998-64f4-4b7d-b6a6-ca198f40af6a service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:02.204471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1328/2652] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:28:02.217573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1325/2653] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:28:02.225753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe926d3-221d-41fb-a202-8a66d18e7f02 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:02.226045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe926d3-221d-41fb-a202-8a66d18e7f02 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:02.226332 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe926d3-221d-41fb-a202-8a66d18e7f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:02.249045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-efe926d3-221d-41fb-a202-8a66d18e7f02 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:02.250253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-efe926d3-221d-41fb-a202-8a66d18e7f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:02.260035 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1329/2654] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:02.263440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc72bb7-bc36-479e-9c28-338167fb3365 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:02.270641 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:02.275643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc72bb7-bc36-479e-9c28-338167fb3365 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:02.282998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18efb2-f14d-4a48-9959-135f1cc82212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:02.283898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18efb2-f14d-4a48-9959-135f1cc82212 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:02.284441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18efb2-f14d-4a48-9959-135f1cc82212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:02.284988 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18efb2-f14d-4a48-9959-135f1cc82212 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:02.285727 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18efb2-f14d-4a48-9959-135f1cc82212 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:28:02.286039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18efb2-f14d-4a48-9959-135f1cc82212 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:02.287700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18efb2-f14d-4a48-9959-135f1cc82212 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:02.302700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1326/2655] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:28:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:28:02.306784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe926d3-221d-41fb-a202-8a66d18e7f02 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:02.319893 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe926d3-221d-41fb-a202-8a66d18e7f02 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:02.325732 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f8d54-021a-4716-9a9c-07b4f0481336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:02.326378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f8d54-021a-4716-9a9c-07b4f0481336 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:02.326674 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f8d54-021a-4716-9a9c-07b4f0481336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:02.326969 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f8d54-021a-4716-9a9c-07b4f0481336 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:02.327678 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f8d54-021a-4716-9a9c-07b4f0481336 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:02.329785 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f8d54-021a-4716-9a9c-07b4f0481336 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:02.365969 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1330/2656] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:28:02 2022] POST /identity/v3/projects => generated 366 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:02.369651 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18efb2-f14d-4a48-9959-135f1cc82212 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:02.383389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18efb2-f14d-4a48-9959-135f1cc82212 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:02.445275 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341426b-ba46-4cbd-9d2e-e8fd4dab8953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:02.445893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341426b-ba46-4cbd-9d2e-e8fd4dab8953 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:02.446181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341426b-ba46-4cbd-9d2e-e8fd4dab8953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:02.446462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341426b-ba46-4cbd-9d2e-e8fd4dab8953 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:02.447173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341426b-ba46-4cbd-9d2e-e8fd4dab8953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:02.449693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341426b-ba46-4cbd-9d2e-e8fd4dab8953 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:02.479251 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1327/2657] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:28:02 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:02.482960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f8d54-021a-4716-9a9c-07b4f0481336 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:02.493397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f8d54-021a-4716-9a9c-07b4f0481336 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:02.498405 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de90bb-63b0-4d0b-8234-fe5904e6dfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:02.499059 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de90bb-63b0-4d0b-8234-fe5904e6dfb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:02.499341 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de90bb-63b0-4d0b-8234-fe5904e6dfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:02.499613 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de90bb-63b0-4d0b-8234-fe5904e6dfb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:02.500331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de90bb-63b0-4d0b-8234-fe5904e6dfb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:02.501979 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de90bb-63b0-4d0b-8234-fe5904e6dfb7 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:02.512490 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1331/2658] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:02 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:02.516594 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341426b-ba46-4cbd-9d2e-e8fd4dab8953 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:02.527891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341426b-ba46-4cbd-9d2e-e8fd4dab8953 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:02.532669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825eac5-c16b-4951-8899-11db705901a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:02.533285 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825eac5-c16b-4951-8899-11db705901a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:02.533567 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825eac5-c16b-4951-8899-11db705901a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:02.533844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825eac5-c16b-4951-8899-11db705901a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:02.534535 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825eac5-c16b-4951-8899-11db705901a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:02.539707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825eac5-c16b-4951-8899-11db705901a1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:02.550480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1328/2659] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:02 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:02.551722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de90bb-63b0-4d0b-8234-fe5904e6dfb7 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:02.562294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de90bb-63b0-4d0b-8234-fe5904e6dfb7 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:02.567595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39dde72-7bbd-49bd-b68a-0fc40c4ba6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:02.568274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dde72-7bbd-49bd-b68a-0fc40c4ba6e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:02.568589 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dde72-7bbd-49bd-b68a-0fc40c4ba6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:02.568866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dde72-7bbd-49bd-b68a-0fc40c4ba6e3 admin admin] PATH_INFO: `/v3/projects/25207c51a0ca49bc8db1fdf75820eb26/users/b8d121a1e2a84e5e9055d95cf5ab36b8/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:02.585663 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39dde72-7bbd-49bd-b68a-0fc40c4ba6e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25207c51a0ca49bc8db1fdf75820eb26, user_id=b8d121a1e2a84e5e9055d95cf5ab36b8, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:02.588394 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39dde72-7bbd-49bd-b68a-0fc40c4ba6e3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:02.602712 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1332/2660] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:02 2022] PUT /identity/v3/projects/25207c51a0ca49bc8db1fdf75820eb26/users/b8d121a1e2a84e5e9055d95cf5ab36b8/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:02.618329 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af19dc-d076-4a56-b66e-0d46efb988e0 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:02.618546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af19dc-d076-4a56-b66e-0d46efb988e0 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:02.618851 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af19dc-d076-4a56-b66e-0d46efb988e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:02.649048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-36af19dc-d076-4a56-b66e-0d46efb988e0 None None] MFA Rules not processed for user `b8d121a1e2a84e5e9055d95cf5ab36b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:02.650322 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-36af19dc-d076-4a56-b66e-0d46efb988e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:02.732228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1329/2661] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:02.736656 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39dde72-7bbd-49bd-b68a-0fc40c4ba6e3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:02.814126 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39dde72-7bbd-49bd-b68a-0fc40c4ba6e3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:02.819979 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2491f47-b02f-44dc-bbd6-ef48179236f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:02.820625 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2491f47-b02f-44dc-bbd6-ef48179236f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:02.820907 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2491f47-b02f-44dc-bbd6-ef48179236f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:02.821187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2491f47-b02f-44dc-bbd6-ef48179236f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:02.821887 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2491f47-b02f-44dc-bbd6-ef48179236f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:02.823841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2491f47-b02f-44dc-bbd6-ef48179236f2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:02.854351 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1333/2662] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:28:02 2022] POST /identity/v3/projects => generated 360 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:02.855596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af19dc-d076-4a56-b66e-0d46efb988e0 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:02.867721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af19dc-d076-4a56-b66e-0d46efb988e0 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:02.927382 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2535a6-5594-40c3-820f-e8ef2098b1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:02.928012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2535a6-5594-40c3-820f-e8ef2098b1b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:02.928300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2535a6-5594-40c3-820f-e8ef2098b1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:02.928581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2535a6-5594-40c3-820f-e8ef2098b1b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:02.929260 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2535a6-5594-40c3-820f-e8ef2098b1b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:02.931200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2535a6-5594-40c3-820f-e8ef2098b1b8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:02.939268 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:02.954989 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1330/2663] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:28:02 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:02.959068 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2491f47-b02f-44dc-bbd6-ef48179236f2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:02.965726 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:02.969552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2491f47-b02f-44dc-bbd6-ef48179236f2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:02.975668 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22219da-8837-4012-86a0-d2c550d3b8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:02.976303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22219da-8837-4012-86a0-d2c550d3b8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:02.976583 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22219da-8837-4012-86a0-d2c550d3b8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:02.976854 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22219da-8837-4012-86a0-d2c550d3b8d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:02.977538 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22219da-8837-4012-86a0-d2c550d3b8d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:02.979168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22219da-8837-4012-86a0-d2c550d3b8d8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:02.990559 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1334/2664] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:02 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:02.994826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2535a6-5594-40c3-820f-e8ef2098b1b8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:03.004768 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2535a6-5594-40c3-820f-e8ef2098b1b8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:03.009304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84f2b6-e4f1-420c-9302-0b4f47c150e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:03.009968 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84f2b6-e4f1-420c-9302-0b4f47c150e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:03.010253 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84f2b6-e4f1-420c-9302-0b4f47c150e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:03.010533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84f2b6-e4f1-420c-9302-0b4f47c150e9 admin admin] PATH_INFO: `/v3/projects/63e50d8a715a4bf398ef20b523dc7500/users/5cce4b2dcf6544d1bfc807974104bc98/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:03.025748 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84f2b6-e4f1-420c-9302-0b4f47c150e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e50d8a715a4bf398ef20b523dc7500, user_id=5cce4b2dcf6544d1bfc807974104bc98, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:03.028669 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84f2b6-e4f1-420c-9302-0b4f47c150e9 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:03.048166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1331/2665] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:02 2022] PUT /identity/v3/projects/63e50d8a715a4bf398ef20b523dc7500/users/5cce4b2dcf6544d1bfc807974104bc98/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:03.052761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22219da-8837-4012-86a0-d2c550d3b8d8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:03.063118 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22219da-8837-4012-86a0-d2c550d3b8d8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:03.122583 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fdbd2-9899-40b0-8bef-6e7b2642136b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:03.123453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fdbd2-9899-40b0-8bef-6e7b2642136b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:03.123740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fdbd2-9899-40b0-8bef-6e7b2642136b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:03.124019 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fdbd2-9899-40b0-8bef-6e7b2642136b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:03.124717 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fdbd2-9899-40b0-8bef-6e7b2642136b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:03.126328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fdbd2-9899-40b0-8bef-6e7b2642136b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:03.136312 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1335/2666] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:03 2022] GET /identity/v3/roles => generated 1427 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:03.140634 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84f2b6-e4f1-420c-9302-0b4f47c150e9 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:03.150968 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84f2b6-e4f1-420c-9302-0b4f47c150e9 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:03.155605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bcc2ad-30e0-4b63-9b1e-edbcb6886db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:03.156258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcc2ad-30e0-4b63-9b1e-edbcb6886db1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:03.156542 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcc2ad-30e0-4b63-9b1e-edbcb6886db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:03.156823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bcc2ad-30e0-4b63-9b1e-edbcb6886db1 admin admin] PATH_INFO: `/v3/projects/63e50d8a715a4bf398ef20b523dc7500/users/5cce4b2dcf6544d1bfc807974104bc98/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:03.158947 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bcc2ad-30e0-4b63-9b1e-edbcb6886db1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e50d8a715a4bf398ef20b523dc7500, user_id=5cce4b2dcf6544d1bfc807974104bc98, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:03.162219 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bcc2ad-30e0-4b63-9b1e-edbcb6886db1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:03.172519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1332/2667] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:03 2022] PUT /identity/v3/projects/63e50d8a715a4bf398ef20b523dc7500/users/5cce4b2dcf6544d1bfc807974104bc98/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:03.183280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f246c6-a435-4c64-ad0d-136ce962d33c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:03.183575 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f246c6-a435-4c64-ad0d-136ce962d33c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:03.183860 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f246c6-a435-4c64-ad0d-136ce962d33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:03.211168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d7f246c6-a435-4c64-ad0d-136ce962d33c None None] MFA Rules not processed for user `5cce4b2dcf6544d1bfc807974104bc98`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:03.212429 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d7f246c6-a435-4c64-ad0d-136ce962d33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:03.278798 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1336/2668] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:03 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:03.291723 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bcc2ad-30e0-4b63-9b1e-edbcb6886db1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:03.293481 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-14bcc2ad-30e0-4b63-9b1e-edbcb6886db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:03.353807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bcc2ad-30e0-4b63-9b1e-edbcb6886db1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:03.357510 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6ed1d-e0fd-4210-a4bd-7a4e83676d3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:03.358121 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6ed1d-e0fd-4210-a4bd-7a4e83676d3c service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:03.358404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6ed1d-e0fd-4210-a4bd-7a4e83676d3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:03.358702 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6ed1d-e0fd-4210-a4bd-7a4e83676d3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:03.368250 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6ed1d-e0fd-4210-a4bd-7a4e83676d3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:03.369676 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6ed1d-e0fd-4210-a4bd-7a4e83676d3c service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:03.379066 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1333/2669] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:03 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:03.809294 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f246c6-a435-4c64-ad0d-136ce962d33c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:03.874154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f246c6-a435-4c64-ad0d-136ce962d33c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:03.878630 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57f889-7408-49bb-9706-3e4fdc63e149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:03.879917 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57f889-7408-49bb-9706-3e4fdc63e149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:03.880198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57f889-7408-49bb-9706-3e4fdc63e149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:03.880475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57f889-7408-49bb-9706-3e4fdc63e149 admin admin] PATH_INFO: `/v3/users/b8d121a1e2a84e5e9055d95cf5ab36b8` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:03.882284 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57f889-7408-49bb-9706-3e4fdc63e149 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d121a1e2a84e5e9055d95cf5ab36b8)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:03.883997 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57f889-7408-49bb-9706-3e4fdc63e149 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:03.899846 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:03.932236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b57f889-7408-49bb-9706-3e4fdc63e149 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d121a1e2a84e5e9055d95cf5ab36b8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:03.932837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b57f889-7408-49bb-9706-3e4fdc63e149 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d121a1e2a84e5e9055d95cf5ab36b8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:03.939875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b57f889-7408-49bb-9706-3e4fdc63e149 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d121a1e2a84e5e9055d95cf5ab36b8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:03.940519 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b57f889-7408-49bb-9706-3e4fdc63e149 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d121a1e2a84e5e9055d95cf5ab36b8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:03.953276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2b57f889-7408-49bb-9706-3e4fdc63e149 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d121a1e2a84e5e9055d95cf5ab36b8'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:03.967834 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1337/2670] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:28:03 2022] DELETE /identity/v3/users/b8d121a1e2a84e5e9055d95cf5ab36b8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:28:03.972855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84c082-a700-465c-adca-6de5369052c9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:03.973419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84c082-a700-465c-adca-6de5369052c9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:03.973419 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84c082-a700-465c-adca-6de5369052c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:03.992588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-eb84c082-a700-465c-adca-6de5369052c9 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:03.993809 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-eb84c082-a700-465c-adca-6de5369052c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:04.055762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1334/2671] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:28:04.065744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57f889-7408-49bb-9706-3e4fdc63e149 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:04.067852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2b57f889-7408-49bb-9706-3e4fdc63e149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:04.128632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57f889-7408-49bb-9706-3e4fdc63e149 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:04.132938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc962a9e-a5d6-4a07-9317-20091fc2f4ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:04.133561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc962a9e-a5d6-4a07-9317-20091fc2f4ba service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:04.133838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc962a9e-a5d6-4a07-9317-20091fc2f4ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:04.134109 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc962a9e-a5d6-4a07-9317-20091fc2f4ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:04.143258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc962a9e-a5d6-4a07-9317-20091fc2f4ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:04.144724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc962a9e-a5d6-4a07-9317-20091fc2f4ba service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:04.154251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1338/2672] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:04.452695 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84c082-a700-465c-adca-6de5369052c9 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:04.454595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-eb84c082-a700-465c-adca-6de5369052c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:04.485256 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84c082-a700-465c-adca-6de5369052c9 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:04.496578 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:04.497234 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:04.497519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:04.497864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] PATH_INFO: `/v3/projects/25207c51a0ca49bc8db1fdf75820eb26` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:04.499266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=25207c51a0ca49bc8db1fdf75820eb26)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:04.501193 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:04.560148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25207c51a0ca49bc8db1fdf75820eb26', 'request_id': 'req-d6237d66-6764-45a6-9a89-d37456b6ef7f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:04.561262 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:04.569120 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25207c51a0ca49bc8db1fdf75820eb26', 'request_id': 'req-d6237d66-6764-45a6-9a89-d37456b6ef7f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:04.579290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25207c51a0ca49bc8db1fdf75820eb26', 'request_id': 'req-d6237d66-6764-45a6-9a89-d37456b6ef7f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:04.591297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] The token cache is being invalidate because project 25207c51a0ca49bc8db1fdf75820eb26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:28:04.591589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:04.592213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:04.593725 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1335/2673] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:28:04 2022] DELETE /identity/v3/projects/25207c51a0ca49bc8db1fdf75820eb26 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:04.598255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc962a9e-a5d6-4a07-9317-20091fc2f4ba service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:04.607942 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc962a9e-a5d6-4a07-9317-20091fc2f4ba service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:04.664628 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1535f2-176c-47bd-87a6-0d9367701bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:04.665265 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1535f2-176c-47bd-87a6-0d9367701bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:04.665552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1535f2-176c-47bd-87a6-0d9367701bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:04.665838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1535f2-176c-47bd-87a6-0d9367701bfe admin admin] PATH_INFO: `/v3/users/5cce4b2dcf6544d1bfc807974104bc98` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:04.667039 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1535f2-176c-47bd-87a6-0d9367701bfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cce4b2dcf6544d1bfc807974104bc98)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:04.669016 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1535f2-176c-47bd-87a6-0d9367701bfe admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:04.708929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:04.715566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd1535f2-176c-47bd-87a6-0d9367701bfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cce4b2dcf6544d1bfc807974104bc98'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:04.716614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd1535f2-176c-47bd-87a6-0d9367701bfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cce4b2dcf6544d1bfc807974104bc98'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:04.723856 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd1535f2-176c-47bd-87a6-0d9367701bfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cce4b2dcf6544d1bfc807974104bc98'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:04.724487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd1535f2-176c-47bd-87a6-0d9367701bfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cce4b2dcf6544d1bfc807974104bc98'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:04.736361 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-fd1535f2-176c-47bd-87a6-0d9367701bfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cce4b2dcf6544d1bfc807974104bc98'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:04.750884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1339/2674] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:28:04 2022] DELETE /identity/v3/users/5cce4b2dcf6544d1bfc807974104bc98 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:05.019182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:05.021826 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:05.038396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6237d66-6764-45a6-9a89-d37456b6ef7f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:05.101183 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd9b36f-5d1f-491f-831b-1fc080161c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:05.101834 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9b36f-5d1f-491f-831b-1fc080161c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:05.102118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9b36f-5d1f-491f-831b-1fc080161c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:05.102395 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd9b36f-5d1f-491f-831b-1fc080161c3f admin admin] PATH_INFO: `/v3/projects/63e50d8a715a4bf398ef20b523dc7500` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:05.103600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9b36f-5d1f-491f-831b-1fc080161c3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=63e50d8a715a4bf398ef20b523dc7500)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:05.105537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd9b36f-5d1f-491f-831b-1fc080161c3f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:05.152619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9cd9b36f-5d1f-491f-831b-1fc080161c3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63e50d8a715a4bf398ef20b523dc7500', 'request_id': 'req-9cd9b36f-5d1f-491f-831b-1fc080161c3f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:05.158919 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9cd9b36f-5d1f-491f-831b-1fc080161c3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63e50d8a715a4bf398ef20b523dc7500', 'request_id': 'req-9cd9b36f-5d1f-491f-831b-1fc080161c3f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:05.166514 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9cd9b36f-5d1f-491f-831b-1fc080161c3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63e50d8a715a4bf398ef20b523dc7500', 'request_id': 'req-9cd9b36f-5d1f-491f-831b-1fc080161c3f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:05.177549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9cd9b36f-5d1f-491f-831b-1fc080161c3f admin admin] The token cache is being invalidate because project 63e50d8a715a4bf398ef20b523dc7500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:28:05.177846 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9cd9b36f-5d1f-491f-831b-1fc080161c3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:05.178500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-9cd9b36f-5d1f-491f-831b-1fc080161c3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:05.180037 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1336/2675] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:28:05 2022] DELETE /identity/v3/projects/63e50d8a715a4bf398ef20b523dc7500 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:05.185143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b300f-fe21-4ab0-8115-d8c3cb9a3aa6 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:05.185639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b300f-fe21-4ab0-8115-d8c3cb9a3aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:05.185744 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b300f-fe21-4ab0-8115-d8c3cb9a3aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:05.205851 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-088b300f-fe21-4ab0-8115-d8c3cb9a3aa6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:05.207093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-088b300f-fe21-4ab0-8115-d8c3cb9a3aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:05.268753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1340/2676] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:05.276753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52172700-3c05-4a9e-89e1-160b14f1050a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:05.277043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52172700-3c05-4a9e-89e1-160b14f1050a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:05.277321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52172700-3c05-4a9e-89e1-160b14f1050a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:05.298784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-52172700-3c05-4a9e-89e1-160b14f1050a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:05.300010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-52172700-3c05-4a9e-89e1-160b14f1050a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:05.308907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1337/2677] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:05.313159 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b300f-fe21-4ab0-8115-d8c3cb9a3aa6 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:05.323876 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b300f-fe21-4ab0-8115-d8c3cb9a3aa6 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:05.328782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e8c12-312e-4279-b9e9-bff567a0a420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:05.329400 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e8c12-312e-4279-b9e9-bff567a0a420 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:05.329685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e8c12-312e-4279-b9e9-bff567a0a420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:05.329964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e8c12-312e-4279-b9e9-bff567a0a420 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:05.330689 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e8c12-312e-4279-b9e9-bff567a0a420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:28:05.331002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e8c12-312e-4279-b9e9-bff567a0a420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:05.332639 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e8c12-312e-4279-b9e9-bff567a0a420 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:05.346092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1341/2678] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:28:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:28:05.351384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52172700-3c05-4a9e-89e1-160b14f1050a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:05.360847 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52172700-3c05-4a9e-89e1-160b14f1050a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:05.366352 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1daa1-e77c-4376-b30b-4d895b0e58b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:05.366991 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1daa1-e77c-4376-b30b-4d895b0e58b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:05.367275 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1daa1-e77c-4376-b30b-4d895b0e58b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:05.367552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1daa1-e77c-4376-b30b-4d895b0e58b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:05.368233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1daa1-e77c-4376-b30b-4d895b0e58b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:05.370136 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1daa1-e77c-4376-b30b-4d895b0e58b1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:05.396832 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1338/2679] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:28:05 2022] POST /identity/v3/projects => generated 370 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:05.400889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e8c12-312e-4279-b9e9-bff567a0a420 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:05.411425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e8c12-312e-4279-b9e9-bff567a0a420 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:05.465993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794afba2-caa3-4048-bfc7-a3d37e70e460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:05.466690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794afba2-caa3-4048-bfc7-a3d37e70e460 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:05.467060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794afba2-caa3-4048-bfc7-a3d37e70e460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:05.467357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794afba2-caa3-4048-bfc7-a3d37e70e460 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:05.468024 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794afba2-caa3-4048-bfc7-a3d37e70e460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:05.469922 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794afba2-caa3-4048-bfc7-a3d37e70e460 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:05.491354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1342/2680] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:28:05 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:05.495473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1daa1-e77c-4376-b30b-4d895b0e58b1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:05.505707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1daa1-e77c-4376-b30b-4d895b0e58b1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:05.510320 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccd50c-7987-4901-8af2-a24a4468e210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:05.510964 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccd50c-7987-4901-8af2-a24a4468e210 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:05.511245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccd50c-7987-4901-8af2-a24a4468e210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:05.511517 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccd50c-7987-4901-8af2-a24a4468e210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:05.512210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccd50c-7987-4901-8af2-a24a4468e210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:05.513928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccd50c-7987-4901-8af2-a24a4468e210 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:05.527469 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1339/2681] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:05 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:05.528617 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794afba2-caa3-4048-bfc7-a3d37e70e460 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:05.539321 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794afba2-caa3-4048-bfc7-a3d37e70e460 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:05.545201 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286aacb1-730b-46b2-894f-5dad60c161a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:05.546111 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aacb1-730b-46b2-894f-5dad60c161a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:05.546408 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aacb1-730b-46b2-894f-5dad60c161a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:05.546712 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aacb1-730b-46b2-894f-5dad60c161a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:05.547427 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286aacb1-730b-46b2-894f-5dad60c161a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:05.549045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286aacb1-730b-46b2-894f-5dad60c161a2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:05.561597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1343/2682] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:05 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:05.565199 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccd50c-7987-4901-8af2-a24a4468e210 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:05.576502 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccd50c-7987-4901-8af2-a24a4468e210 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:05.581312 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607b647-3298-46ec-a8a0-df6db7336776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:05.581978 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607b647-3298-46ec-a8a0-df6db7336776 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:05.582269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607b647-3298-46ec-a8a0-df6db7336776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:05.582581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607b647-3298-46ec-a8a0-df6db7336776 admin admin] PATH_INFO: `/v3/projects/df282c663e164bd08e5e4ce3c040f3ff/users/dd6037611d554b30b0868627c1d67319/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:05.598803 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607b647-3298-46ec-a8a0-df6db7336776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df282c663e164bd08e5e4ce3c040f3ff, user_id=dd6037611d554b30b0868627c1d67319, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:05.612009 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607b647-3298-46ec-a8a0-df6db7336776 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:05.627899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1340/2683] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:05 2022] PUT /identity/v3/projects/df282c663e164bd08e5e4ce3c040f3ff/users/dd6037611d554b30b0868627c1d67319/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:05.639821 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0ad5a-4a3b-4713-9f24-a2a9a9e9fea3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:05.640114 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0ad5a-4a3b-4713-9f24-a2a9a9e9fea3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:05.640397 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0ad5a-4a3b-4713-9f24-a2a9a9e9fea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:05.669552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-1cf0ad5a-4a3b-4713-9f24-a2a9a9e9fea3 None None] MFA Rules not processed for user `dd6037611d554b30b0868627c1d67319`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:05.670831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-1cf0ad5a-4a3b-4713-9f24-a2a9a9e9fea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:05.680968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:05.756475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1344/2684] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:05.761224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607b647-3298-46ec-a8a0-df6db7336776 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:05.767433 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:05.829082 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607b647-3298-46ec-a8a0-df6db7336776 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:05.840830 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7930e-e116-4f4e-94c0-4bb366001645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:05.841491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7930e-e116-4f4e-94c0-4bb366001645 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:05.842153 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7930e-e116-4f4e-94c0-4bb366001645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:05.842153 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7930e-e116-4f4e-94c0-4bb366001645 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:05.842749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7930e-e116-4f4e-94c0-4bb366001645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:05.844674 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7930e-e116-4f4e-94c0-4bb366001645 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:05.872350 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1341/2685] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:28:05 2022] POST /identity/v3/projects => generated 370 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:28:05.876499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0ad5a-4a3b-4713-9f24-a2a9a9e9fea3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:05.887929 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0ad5a-4a3b-4713-9f24-a2a9a9e9fea3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:05.948235 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77826316-159b-403d-b529-b0a1fe4b3a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:05.948867 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77826316-159b-403d-b529-b0a1fe4b3a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:05.949154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77826316-159b-403d-b529-b0a1fe4b3a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:05.949432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77826316-159b-403d-b529-b0a1fe4b3a31 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:05.950108 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77826316-159b-403d-b529-b0a1fe4b3a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:05.952358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77826316-159b-403d-b529-b0a1fe4b3a31 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:05.974297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1345/2686] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:28:05 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:05.978263 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7930e-e116-4f4e-94c0-4bb366001645 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:05.988494 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7930e-e116-4f4e-94c0-4bb366001645 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:05.993170 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d263f-76c6-4314-a88d-c3ccee7452c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:05.993776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d263f-76c6-4314-a88d-c3ccee7452c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:05.994061 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d263f-76c6-4314-a88d-c3ccee7452c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:05.994337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d263f-76c6-4314-a88d-c3ccee7452c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:05.995107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d263f-76c6-4314-a88d-c3ccee7452c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:05.996709 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d263f-76c6-4314-a88d-c3ccee7452c1 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:06.007871 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1342/2687] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:05 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:06.011827 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77826316-159b-403d-b529-b0a1fe4b3a31 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:06.022539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77826316-159b-403d-b529-b0a1fe4b3a31 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:06.027335 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b32deb-6a79-4976-8ca2-ec63fb9622ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:06.028001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b32deb-6a79-4976-8ca2-ec63fb9622ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:06.028284 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b32deb-6a79-4976-8ca2-ec63fb9622ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:06.028573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b32deb-6a79-4976-8ca2-ec63fb9622ff admin admin] PATH_INFO: `/v3/projects/d7ab167051784c899f2e54100d257c53/users/e3cbf1ab637e4732b6eb247bb7445cf0/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:06.043306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b32deb-6a79-4976-8ca2-ec63fb9622ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ab167051784c899f2e54100d257c53, user_id=e3cbf1ab637e4732b6eb247bb7445cf0, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:06.045960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b32deb-6a79-4976-8ca2-ec63fb9622ff admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:06.058769 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1346/2688] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:06 2022] PUT /identity/v3/projects/d7ab167051784c899f2e54100d257c53/users/e3cbf1ab637e4732b6eb247bb7445cf0/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:06.062893 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d263f-76c6-4314-a88d-c3ccee7452c1 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:06.073944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d263f-76c6-4314-a88d-c3ccee7452c1 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:06.134681 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8378b7f-1a01-45c3-b508-4de74a43f2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:06.135307 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8378b7f-1a01-45c3-b508-4de74a43f2aa admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:06.135587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8378b7f-1a01-45c3-b508-4de74a43f2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:06.135857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8378b7f-1a01-45c3-b508-4de74a43f2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:06.136549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8378b7f-1a01-45c3-b508-4de74a43f2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:06.138168 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8378b7f-1a01-45c3-b508-4de74a43f2aa admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:06.148805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1343/2689] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:06 2022] GET /identity/v3/roles => generated 1427 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:06.153010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b32deb-6a79-4976-8ca2-ec63fb9622ff admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:06.163249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b32deb-6a79-4976-8ca2-ec63fb9622ff admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:06.168000 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401febf8-04b4-4708-89b7-0b9964b0231b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:06.168658 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401febf8-04b4-4708-89b7-0b9964b0231b admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:06.168941 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401febf8-04b4-4708-89b7-0b9964b0231b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:06.169246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401febf8-04b4-4708-89b7-0b9964b0231b admin admin] PATH_INFO: `/v3/projects/d7ab167051784c899f2e54100d257c53/users/e3cbf1ab637e4732b6eb247bb7445cf0/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:06.171283 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401febf8-04b4-4708-89b7-0b9964b0231b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ab167051784c899f2e54100d257c53, user_id=e3cbf1ab637e4732b6eb247bb7445cf0, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:06.173912 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401febf8-04b4-4708-89b7-0b9964b0231b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:06.184079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1347/2690] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:06 2022] PUT /identity/v3/projects/d7ab167051784c899f2e54100d257c53/users/e3cbf1ab637e4732b6eb247bb7445cf0/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:06.194767 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49b238-158d-4c31-b7a9-6fea92fedbc6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:06.195071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49b238-158d-4c31-b7a9-6fea92fedbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:06.195347 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49b238-158d-4c31-b7a9-6fea92fedbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:06.223499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1a49b238-158d-4c31-b7a9-6fea92fedbc6 None None] MFA Rules not processed for user `e3cbf1ab637e4732b6eb247bb7445cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:06.224715 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1a49b238-158d-4c31-b7a9-6fea92fedbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:06.289166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:06.293296 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1344/2691] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:06.303692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401febf8-04b4-4708-89b7-0b9964b0231b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:06.305399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-401febf8-04b4-4708-89b7-0b9964b0231b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:06.365709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401febf8-04b4-4708-89b7-0b9964b0231b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:06.369408 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a297c-9cd1-49e4-84d7-462167c63d49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:06.370025 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a297c-9cd1-49e4-84d7-462167c63d49 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:06.370306 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a297c-9cd1-49e4-84d7-462167c63d49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:06.370581 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a297c-9cd1-49e4-84d7-462167c63d49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:06.380013 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a297c-9cd1-49e4-84d7-462167c63d49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:06.381446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a297c-9cd1-49e4-84d7-462167c63d49 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:06.390801 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1348/2692] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:07.733753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49b238-158d-4c31-b7a9-6fea92fedbc6 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:07.796221 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49b238-158d-4c31-b7a9-6fea92fedbc6 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:07.801197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3a95d-7d2f-4bfe-b660-659c40e67f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:07.801849 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3a95d-7d2f-4bfe-b660-659c40e67f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:07.802134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3a95d-7d2f-4bfe-b660-659c40e67f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:07.802435 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3a95d-7d2f-4bfe-b660-659c40e67f28 admin admin] PATH_INFO: `/v3/users/dd6037611d554b30b0868627c1d67319` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:07.803660 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3a95d-7d2f-4bfe-b660-659c40e67f28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd6037611d554b30b0868627c1d67319)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:07.805603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3a95d-7d2f-4bfe-b660-659c40e67f28 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:07.854206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3eb3a95d-7d2f-4bfe-b660-659c40e67f28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd6037611d554b30b0868627c1d67319'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:07.864219 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3eb3a95d-7d2f-4bfe-b660-659c40e67f28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd6037611d554b30b0868627c1d67319'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:07.864896 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3eb3a95d-7d2f-4bfe-b660-659c40e67f28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd6037611d554b30b0868627c1d67319'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:07.865712 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3eb3a95d-7d2f-4bfe-b660-659c40e67f28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd6037611d554b30b0868627c1d67319'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:07.878271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-3eb3a95d-7d2f-4bfe-b660-659c40e67f28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd6037611d554b30b0868627c1d67319'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:07.891271 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1345/2693] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:28:07 2022] DELETE /identity/v3/users/dd6037611d554b30b0868627c1d67319 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:07.895831 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43bf1-d822-4bc6-af81-97384d856059 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:07.896312 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43bf1-d822-4bc6-af81-97384d856059 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:07.896429 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f43bf1-d822-4bc6-af81-97384d856059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:07.911105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:07.917740 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-59f43bf1-d822-4bc6-af81-97384d856059 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:07.918991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-59f43bf1-d822-4bc6-af81-97384d856059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:07.981647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1349/2694] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:07.992781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3a95d-7d2f-4bfe-b660-659c40e67f28 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:07.994321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-3eb3a95d-7d2f-4bfe-b660-659c40e67f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:08.054440 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3a95d-7d2f-4bfe-b660-659c40e67f28 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:08.058140 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d10b49-2651-4064-b77f-2c1771b52cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:08.058786 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d10b49-2651-4064-b77f-2c1771b52cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:08.059074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d10b49-2651-4064-b77f-2c1771b52cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:08.059353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d10b49-2651-4064-b77f-2c1771b52cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:08.069218 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d10b49-2651-4064-b77f-2c1771b52cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:08.070670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d10b49-2651-4064-b77f-2c1771b52cc6 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:08.080527 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1346/2695] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:08.366098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f43bf1-d822-4bc6-af81-97384d856059 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:08.367578 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-59f43bf1-d822-4bc6-af81-97384d856059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:08.383982 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f43bf1-d822-4bc6-af81-97384d856059 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:08.390523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:08.391372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:08.391727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:08.392069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] PATH_INFO: `/v3/projects/df282c663e164bd08e5e4ce3c040f3ff` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:08.393339 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df282c663e164bd08e5e4ce3c040f3ff)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:08.395940 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:08.450791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df282c663e164bd08e5e4ce3c040f3ff', 'request_id': 'req-7464ad7c-d1c4-4508-904c-1aa99a87be47'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:08.457100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df282c663e164bd08e5e4ce3c040f3ff', 'request_id': 'req-7464ad7c-d1c4-4508-904c-1aa99a87be47'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:08.463403 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df282c663e164bd08e5e4ce3c040f3ff', 'request_id': 'req-7464ad7c-d1c4-4508-904c-1aa99a87be47'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:08.476813 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] The token cache is being invalidate because project df282c663e164bd08e5e4ce3c040f3ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:28:08.477110 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:08.477743 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:08.479327 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1350/2696] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:28:08 2022] DELETE /identity/v3/projects/df282c663e164bd08e5e4ce3c040f3ff => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:08.483373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d10b49-2651-4064-b77f-2c1771b52cc6 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:08.493838 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d10b49-2651-4064-b77f-2c1771b52cc6 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:08.512624 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:08.554584 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a0589-f2c8-48bd-842a-431367923c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:08.555269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a0589-f2c8-48bd-842a-431367923c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:08.555583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a0589-f2c8-48bd-842a-431367923c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:08.555881 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a0589-f2c8-48bd-842a-431367923c23 admin admin] PATH_INFO: `/v3/users/e3cbf1ab637e4732b6eb247bb7445cf0` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:08.557124 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a0589-f2c8-48bd-842a-431367923c23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3cbf1ab637e4732b6eb247bb7445cf0)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:08.559139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a0589-f2c8-48bd-842a-431367923c23 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:08.605952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-af3a0589-f2c8-48bd-842a-431367923c23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3cbf1ab637e4732b6eb247bb7445cf0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:08.614728 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-af3a0589-f2c8-48bd-842a-431367923c23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3cbf1ab637e4732b6eb247bb7445cf0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:08.615555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-af3a0589-f2c8-48bd-842a-431367923c23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3cbf1ab637e4732b6eb247bb7445cf0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:08.616205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-af3a0589-f2c8-48bd-842a-431367923c23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3cbf1ab637e4732b6eb247bb7445cf0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:08.628849 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-af3a0589-f2c8-48bd-842a-431367923c23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3cbf1ab637e4732b6eb247bb7445cf0'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:08.645900 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1347/2697] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:28:08 2022] DELETE /identity/v3/users/e3cbf1ab637e4732b6eb247bb7445cf0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:08.979859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:08.982187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:09.003220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464ad7c-d1c4-4508-904c-1aa99a87be47 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:09.062774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93348ec5-cd75-4491-9bd4-0b21d815e78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:09.063428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93348ec5-cd75-4491-9bd4-0b21d815e78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:09.063708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93348ec5-cd75-4491-9bd4-0b21d815e78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:09.063986 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93348ec5-cd75-4491-9bd4-0b21d815e78c admin admin] PATH_INFO: `/v3/projects/d7ab167051784c899f2e54100d257c53` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:09.065152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93348ec5-cd75-4491-9bd4-0b21d815e78c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7ab167051784c899f2e54100d257c53)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:09.067880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93348ec5-cd75-4491-9bd4-0b21d815e78c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:09.103783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:09.123356 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-93348ec5-cd75-4491-9bd4-0b21d815e78c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ab167051784c899f2e54100d257c53', 'request_id': 'req-93348ec5-cd75-4491-9bd4-0b21d815e78c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:09.130990 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-93348ec5-cd75-4491-9bd4-0b21d815e78c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ab167051784c899f2e54100d257c53', 'request_id': 'req-93348ec5-cd75-4491-9bd4-0b21d815e78c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:09.137295 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-93348ec5-cd75-4491-9bd4-0b21d815e78c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ab167051784c899f2e54100d257c53', 'request_id': 'req-93348ec5-cd75-4491-9bd4-0b21d815e78c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:09.151548 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-93348ec5-cd75-4491-9bd4-0b21d815e78c admin admin] The token cache is being invalidate because project d7ab167051784c899f2e54100d257c53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:28:09.151968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-93348ec5-cd75-4491-9bd4-0b21d815e78c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:09.153002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-93348ec5-cd75-4491-9bd4-0b21d815e78c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:09.154478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1351/2698] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:28:08 2022] DELETE /identity/v3/projects/d7ab167051784c899f2e54100d257c53 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:09.159519 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38b237-9d0c-4981-85ca-f986b9774313 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:09.160102 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38b237-9d0c-4981-85ca-f986b9774313 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:09.160390 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38b237-9d0c-4981-85ca-f986b9774313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:09.184363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-2e38b237-9d0c-4981-85ca-f986b9774313 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:09.185620 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-2e38b237-9d0c-4981-85ca-f986b9774313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:09.254740 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1348/2699] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:28:09.264020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fa22d-78e4-47f3-bcfd-45a11d72e0ed None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:09.264458 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fa22d-78e4-47f3-bcfd-45a11d72e0ed None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:09.265146 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fa22d-78e4-47f3-bcfd-45a11d72e0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:09.300357 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bf9fa22d-78e4-47f3-bcfd-45a11d72e0ed None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:09.301746 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bf9fa22d-78e4-47f3-bcfd-45a11d72e0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:09.310057 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1352/2700] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:09.314938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38b237-9d0c-4981-85ca-f986b9774313 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:09.326148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38b237-9d0c-4981-85ca-f986b9774313 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:09.330976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc62765-3bda-48bc-9652-5cf31d0a59a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:09.331699 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc62765-3bda-48bc-9652-5cf31d0a59a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:09.332143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc62765-3bda-48bc-9652-5cf31d0a59a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:09.333047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc62765-3bda-48bc-9652-5cf31d0a59a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:09.333848 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc62765-3bda-48bc-9652-5cf31d0a59a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:28:09.334222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc62765-3bda-48bc-9652-5cf31d0a59a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:09.335954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc62765-3bda-48bc-9652-5cf31d0a59a7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:09.348798 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1349/2701] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:28:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:28:09.353251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9fa22d-78e4-47f3-bcfd-45a11d72e0ed None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:09.363826 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9fa22d-78e4-47f3-bcfd-45a11d72e0ed None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:09.369518 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e4ad8-b33c-48b1-92e4-15970ebaa3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:09.370154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e4ad8-b33c-48b1-92e4-15970ebaa3b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:09.370432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e4ad8-b33c-48b1-92e4-15970ebaa3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:09.370735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e4ad8-b33c-48b1-92e4-15970ebaa3b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:09.371428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e4ad8-b33c-48b1-92e4-15970ebaa3b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:09.373528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0e4ad8-b33c-48b1-92e4-15970ebaa3b0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:09.410952 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1353/2702] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:28:09 2022] POST /identity/v3/projects => generated 386 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:09.415057 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc62765-3bda-48bc-9652-5cf31d0a59a7 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:09.427595 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc62765-3bda-48bc-9652-5cf31d0a59a7 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:09.462860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:09.501809 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061350cb-4d86-470d-93d4-5327ada6b7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:09.502434 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061350cb-4d86-470d-93d4-5327ada6b7e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:09.502745 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061350cb-4d86-470d-93d4-5327ada6b7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:09.503024 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061350cb-4d86-470d-93d4-5327ada6b7e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:09.503724 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061350cb-4d86-470d-93d4-5327ada6b7e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:09.506310 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061350cb-4d86-470d-93d4-5327ada6b7e6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:09.533598 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1350/2703] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:28:09 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:28:09.534898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e4ad8-b33c-48b1-92e4-15970ebaa3b0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:09.553414 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e4ad8-b33c-48b1-92e4-15970ebaa3b0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:09.559957 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ada2f-2272-45bf-a3f4-79f3243945c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:09.560792 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ada2f-2272-45bf-a3f4-79f3243945c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:09.561080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ada2f-2272-45bf-a3f4-79f3243945c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:09.561358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ada2f-2272-45bf-a3f4-79f3243945c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:09.562124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ada2f-2272-45bf-a3f4-79f3243945c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:09.564140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ada2f-2272-45bf-a3f4-79f3243945c6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:09.583589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061350cb-4d86-470d-93d4-5327ada6b7e6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:09.594768 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1354/2704] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:09 2022] GET /identity/v3/roles => generated 1427 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:09.595085 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061350cb-4d86-470d-93d4-5327ada6b7e6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:09.600399 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eab2dc-c30f-4357-afa4-db159a6872d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:09.601108 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eab2dc-c30f-4357-afa4-db159a6872d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:09.601398 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eab2dc-c30f-4357-afa4-db159a6872d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:09.601871 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eab2dc-c30f-4357-afa4-db159a6872d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:09.602822 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eab2dc-c30f-4357-afa4-db159a6872d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:09.604746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eab2dc-c30f-4357-afa4-db159a6872d7 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:09.624751 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1351/2705] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:09 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:28:09.626140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ada2f-2272-45bf-a3f4-79f3243945c6 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:09.647856 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ada2f-2272-45bf-a3f4-79f3243945c6 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:09.653020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58174561-fa9b-4167-806d-d889af48bec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:09.653815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58174561-fa9b-4167-806d-d889af48bec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:09.654210 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58174561-fa9b-4167-806d-d889af48bec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:09.654545 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58174561-fa9b-4167-806d-d889af48bec4 admin admin] PATH_INFO: `/v3/projects/50af00285d8543f1b1dbd82bf173aa0d/users/95c49b58a5fd4da682a88b27407d2a80/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:09.671561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58174561-fa9b-4167-806d-d889af48bec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50af00285d8543f1b1dbd82bf173aa0d, user_id=95c49b58a5fd4da682a88b27407d2a80, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:09.674522 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58174561-fa9b-4167-806d-d889af48bec4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:09.695796 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1355/2706] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:09 2022] PUT /identity/v3/projects/50af00285d8543f1b1dbd82bf173aa0d/users/95c49b58a5fd4da682a88b27407d2a80/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:09.706448 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c6c7b-e640-426c-b3e6-64b3fd96624c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:09.706780 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c6c7b-e640-426c-b3e6-64b3fd96624c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:09.707603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c6c7b-e640-426c-b3e6-64b3fd96624c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:09.741290 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-200c6c7b-e640-426c-b3e6-64b3fd96624c None None] MFA Rules not processed for user `95c49b58a5fd4da682a88b27407d2a80`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:09.742540 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-200c6c7b-e640-426c-b3e6-64b3fd96624c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:09.943590 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1352/2707] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:09.948266 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58174561-fa9b-4167-806d-d889af48bec4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:10.016157 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58174561-fa9b-4167-806d-d889af48bec4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:10.021595 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18342dd-ead4-4c8f-bce6-9b45c5b19ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:10.022370 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18342dd-ead4-4c8f-bce6-9b45c5b19ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:10.022676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18342dd-ead4-4c8f-bce6-9b45c5b19ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:10.022964 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18342dd-ead4-4c8f-bce6-9b45c5b19ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:10.024514 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18342dd-ead4-4c8f-bce6-9b45c5b19ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:10.027300 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18342dd-ead4-4c8f-bce6-9b45c5b19ceb admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:10.054889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1356/2708] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:28:09 2022] POST /identity/v3/projects => generated 386 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:10.059006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c6c7b-e640-426c-b3e6-64b3fd96624c None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:10.070331 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c6c7b-e640-426c-b3e6-64b3fd96624c None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:10.136484 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9d7bf-b5db-4b91-89f3-d18749746aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:10.137105 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9d7bf-b5db-4b91-89f3-d18749746aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:10.137387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9d7bf-b5db-4b91-89f3-d18749746aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:10.137773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9d7bf-b5db-4b91-89f3-d18749746aa8 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:10.138442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9d7bf-b5db-4b91-89f3-d18749746aa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:10.140428 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9d7bf-b5db-4b91-89f3-d18749746aa8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:10.168592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1353/2709] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:28:10 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:28:10.169197 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18342dd-ead4-4c8f-bce6-9b45c5b19ceb admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:10.179280 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18342dd-ead4-4c8f-bce6-9b45c5b19ceb admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:10.184386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658f2e5-4bfb-45ee-829e-7353e259e748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:10.185013 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658f2e5-4bfb-45ee-829e-7353e259e748 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:10.185299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658f2e5-4bfb-45ee-829e-7353e259e748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:10.185576 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658f2e5-4bfb-45ee-829e-7353e259e748 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:10.186268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658f2e5-4bfb-45ee-829e-7353e259e748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:10.188104 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658f2e5-4bfb-45ee-829e-7353e259e748 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:10.200856 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1357/2710] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:10 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:10.205104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9d7bf-b5db-4b91-89f3-d18749746aa8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:10.218525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9d7bf-b5db-4b91-89f3-d18749746aa8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:10.224043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc0a40-67c8-4749-a6e2-94a69afe940a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:10.224781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0a40-67c8-4749-a6e2-94a69afe940a admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:10.225122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0a40-67c8-4749-a6e2-94a69afe940a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:10.225397 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc0a40-67c8-4749-a6e2-94a69afe940a admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:10.226157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc0a40-67c8-4749-a6e2-94a69afe940a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:10.227970 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc0a40-67c8-4749-a6e2-94a69afe940a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:10.238800 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1354/2711] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:10 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:28:10.243423 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658f2e5-4bfb-45ee-829e-7353e259e748 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:10.254102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658f2e5-4bfb-45ee-829e-7353e259e748 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:10.259165 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2c29d-54a3-4cdf-a588-96e88f10e86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:10.259900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2c29d-54a3-4cdf-a588-96e88f10e86f admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:10.260244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2c29d-54a3-4cdf-a588-96e88f10e86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:10.260581 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2c29d-54a3-4cdf-a588-96e88f10e86f admin admin] PATH_INFO: `/v3/projects/827cf7b37159463aad4177b06560e2ae/users/2af64dd44da14b51805fb0cc78a33c8d/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:10.277392 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2c29d-54a3-4cdf-a588-96e88f10e86f admin admin] RBAC: Authorizing `identity:create_grant(project_id=827cf7b37159463aad4177b06560e2ae, user_id=2af64dd44da14b51805fb0cc78a33c8d, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:10.280185 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2c29d-54a3-4cdf-a588-96e88f10e86f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:10.294276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1358/2712] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:10 2022] PUT /identity/v3/projects/827cf7b37159463aad4177b06560e2ae/users/2af64dd44da14b51805fb0cc78a33c8d/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:10.305101 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6a27b-f3aa-4e58-97e0-e17bb30373c6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:10.305652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6a27b-f3aa-4e58-97e0-e17bb30373c6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:10.305961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6a27b-f3aa-4e58-97e0-e17bb30373c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:10.336048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5cc6a27b-f3aa-4e58-97e0-e17bb30373c6 None None] MFA Rules not processed for user `2af64dd44da14b51805fb0cc78a33c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:10.337460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5cc6a27b-f3aa-4e58-97e0-e17bb30373c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:10.341372 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:10.405832 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1355/2713] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:10.411175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2c29d-54a3-4cdf-a588-96e88f10e86f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:10.422875 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:10.491098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2c29d-54a3-4cdf-a588-96e88f10e86f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:10.495940 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10309080-981d-4bdc-8de5-55122e47462b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:10.496563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10309080-981d-4bdc-8de5-55122e47462b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:10.496851 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10309080-981d-4bdc-8de5-55122e47462b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:10.497133 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10309080-981d-4bdc-8de5-55122e47462b admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:10.497814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10309080-981d-4bdc-8de5-55122e47462b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:10.499773 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10309080-981d-4bdc-8de5-55122e47462b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:10.530622 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1359/2714] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:28:10 2022] POST /identity/v3/projects => generated 386 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:10.535213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6a27b-f3aa-4e58-97e0-e17bb30373c6 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:10.553464 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6a27b-f3aa-4e58-97e0-e17bb30373c6 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:10.621773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbee15-816f-4c3a-b3bc-d20df4104fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:10.622063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbee15-816f-4c3a-b3bc-d20df4104fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:10.622321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbee15-816f-4c3a-b3bc-d20df4104fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:10.622716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbee15-816f-4c3a-b3bc-d20df4104fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:10.623458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbee15-816f-4c3a-b3bc-d20df4104fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:10.625777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbee15-816f-4c3a-b3bc-d20df4104fb3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:10.647955 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1356/2715] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:28:10 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:28:10.652237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10309080-981d-4bdc-8de5-55122e47462b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:10.662959 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10309080-981d-4bdc-8de5-55122e47462b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:10.667566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654f2ab-a63d-4be4-a755-b28b7e521474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:10.668179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654f2ab-a63d-4be4-a755-b28b7e521474 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:10.668458 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654f2ab-a63d-4be4-a755-b28b7e521474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:10.668733 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654f2ab-a63d-4be4-a755-b28b7e521474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:10.669532 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654f2ab-a63d-4be4-a755-b28b7e521474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:10.671175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654f2ab-a63d-4be4-a755-b28b7e521474 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:10.682877 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1360/2716] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:10 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:10.687362 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbee15-816f-4c3a-b3bc-d20df4104fb3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:10.697929 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbee15-816f-4c3a-b3bc-d20df4104fb3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:10.703049 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799ed69-e16d-4215-b8f4-96235edd1343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:10.703697 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799ed69-e16d-4215-b8f4-96235edd1343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:10.703978 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799ed69-e16d-4215-b8f4-96235edd1343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:10.704249 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799ed69-e16d-4215-b8f4-96235edd1343 admin admin] PATH_INFO: `/v3/projects/d6cf7d896f8441e199ff094bfa5b3c6b/users/506d254ce58c4fe68fc6a6a04f9821e2/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:10.722007 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3799ed69-e16d-4215-b8f4-96235edd1343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6cf7d896f8441e199ff094bfa5b3c6b, user_id=506d254ce58c4fe68fc6a6a04f9821e2, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:10.724823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3799ed69-e16d-4215-b8f4-96235edd1343 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:10.737952 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1357/2717] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:10 2022] PUT /identity/v3/projects/d6cf7d896f8441e199ff094bfa5b3c6b/users/506d254ce58c4fe68fc6a6a04f9821e2/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:28:10.742150 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654f2ab-a63d-4be4-a755-b28b7e521474 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:10.753562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654f2ab-a63d-4be4-a755-b28b7e521474 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:10.815154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bb056-9236-4024-9fc4-73fe36ad2566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:10.815697 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bb056-9236-4024-9fc4-73fe36ad2566 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:10.815984 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bb056-9236-4024-9fc4-73fe36ad2566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:10.816758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bb056-9236-4024-9fc4-73fe36ad2566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:10.817466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bb056-9236-4024-9fc4-73fe36ad2566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:10.819172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bb056-9236-4024-9fc4-73fe36ad2566 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:10.830051 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1361/2718] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:10 2022] GET /identity/v3/roles => generated 1427 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:10.835347 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799ed69-e16d-4215-b8f4-96235edd1343 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:10.846995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799ed69-e16d-4215-b8f4-96235edd1343 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:10.851693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35090328-543a-41ce-bead-324c066426cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:10.852359 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35090328-543a-41ce-bead-324c066426cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:10.852644 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35090328-543a-41ce-bead-324c066426cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:10.852928 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35090328-543a-41ce-bead-324c066426cc admin admin] PATH_INFO: `/v3/projects/d6cf7d896f8441e199ff094bfa5b3c6b/users/506d254ce58c4fe68fc6a6a04f9821e2/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:10.854927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35090328-543a-41ce-bead-324c066426cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6cf7d896f8441e199ff094bfa5b3c6b, user_id=506d254ce58c4fe68fc6a6a04f9821e2, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:10.857576 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35090328-543a-41ce-bead-324c066426cc admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:10.868925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1358/2719] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:10 2022] PUT /identity/v3/projects/d6cf7d896f8441e199ff094bfa5b3c6b/users/506d254ce58c4fe68fc6a6a04f9821e2/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:10.937253 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea58f50-d3db-487e-ac12-09b29e900451 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:10.937549 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea58f50-d3db-487e-ac12-09b29e900451 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:10.937832 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea58f50-d3db-487e-ac12-09b29e900451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:10.967811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6ea58f50-d3db-487e-ac12-09b29e900451 None None] MFA Rules not processed for user `506d254ce58c4fe68fc6a6a04f9821e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:10.969390 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6ea58f50-d3db-487e-ac12-09b29e900451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:11.041164 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1362/2720] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:11.052866 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35090328-543a-41ce-bead-324c066426cc admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:11.054649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-35090328-543a-41ce-bead-324c066426cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:11.114397 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35090328-543a-41ce-bead-324c066426cc admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:11.118377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df574e1d-2ebe-4b06-9318-b2d7589e8c57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:11.119176 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df574e1d-2ebe-4b06-9318-b2d7589e8c57 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:11.119456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df574e1d-2ebe-4b06-9318-b2d7589e8c57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:11.119729 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df574e1d-2ebe-4b06-9318-b2d7589e8c57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:11.129506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df574e1d-2ebe-4b06-9318-b2d7589e8c57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:11.130962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df574e1d-2ebe-4b06-9318-b2d7589e8c57 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:11.135974 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:11.142354 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1359/2721] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:11.403386 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fb022-54ab-46ec-bbe4-95b9b029a34a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:11.403698 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fb022-54ab-46ec-bbe4-95b9b029a34a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:11.403974 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9fb022-54ab-46ec-bbe4-95b9b029a34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:11.411661 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:11.427724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-4b9fb022-54ab-46ec-bbe4-95b9b029a34a None None] MFA Rules not processed for user `49f98078ac424b1284932839be652265`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:11.428946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4b9fb022-54ab-46ec-bbe4-95b9b029a34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:11.490240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1363/2722] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:11.672901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df574e1d-2ebe-4b06-9318-b2d7589e8c57 service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:11.675218 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-df574e1d-2ebe-4b06-9318-b2d7589e8c57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:11.687242 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df574e1d-2ebe-4b06-9318-b2d7589e8c57 service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:11.693581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba873ef8-9711-4dc1-9112-56f8fa5e1d4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:11.694220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873ef8-9711-4dc1-9112-56f8fa5e1d4d service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:11.694501 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873ef8-9711-4dc1-9112-56f8fa5e1d4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:11.694813 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873ef8-9711-4dc1-9112-56f8fa5e1d4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:11.753477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba873ef8-9711-4dc1-9112-56f8fa5e1d4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:11.755004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba873ef8-9711-4dc1-9112-56f8fa5e1d4d service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:11.764015 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1360/2723] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:11.820093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fb022-54ab-46ec-bbe4-95b9b029a34a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:11.821372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4b9fb022-54ab-46ec-bbe4-95b9b029a34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:11.881842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9fb022-54ab-46ec-bbe4-95b9b029a34a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:11.885383 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb35266-1e2a-4d86-8f91-3bcaa9ec4216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:11.886314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb35266-1e2a-4d86-8f91-3bcaa9ec4216 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:11.886314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb35266-1e2a-4d86-8f91-3bcaa9ec4216 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:11.886535 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb35266-1e2a-4d86-8f91-3bcaa9ec4216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:11.888363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bfb35266-1e2a-4d86-8f91-3bcaa9ec4216 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:11.899235 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb35266-1e2a-4d86-8f91-3bcaa9ec4216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:11.900709 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb35266-1e2a-4d86-8f91-3bcaa9ec4216 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:11.909260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1364/2724] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:12.245196 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba873ef8-9711-4dc1-9112-56f8fa5e1d4d service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:12.246615 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ba873ef8-9711-4dc1-9112-56f8fa5e1d4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:12.320872 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba873ef8-9711-4dc1-9112-56f8fa5e1d4d service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:12.326149 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a3de3-7cf6-4f28-a8c1-b32619fcb1b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:12.326790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a3de3-7cf6-4f28-a8c1-b32619fcb1b2 service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:12.327077 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a3de3-7cf6-4f28-a8c1-b32619fcb1b2 service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:12.327354 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a3de3-7cf6-4f28-a8c1-b32619fcb1b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:12.329122 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c28a3de3-7cf6-4f28-a8c1-b32619fcb1b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:12.340975 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a3de3-7cf6-4f28-a8c1-b32619fcb1b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:12.342559 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a3de3-7cf6-4f28-a8c1-b32619fcb1b2 service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:12.352341 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1361/2725] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:15.852791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb35266-1e2a-4d86-8f91-3bcaa9ec4216 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:15.862380 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb35266-1e2a-4d86-8f91-3bcaa9ec4216 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:15.865976 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd0afb-2514-4d64-9700-0abcc5ba039d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:15.866608 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0afb-2514-4d64-9700-0abcc5ba039d service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:15.866920 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0afb-2514-4d64-9700-0abcc5ba039d service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:15.867188 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0afb-2514-4d64-9700-0abcc5ba039d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:15.904191 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a3de3-7cf6-4f28-a8c1-b32619fcb1b2 service placement] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:15.905604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c28a3de3-7cf6-4f28-a8c1-b32619fcb1b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:15.924927 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd0afb-2514-4d64-9700-0abcc5ba039d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:15.926647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd0afb-2514-4d64-9700-0abcc5ba039d service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:15.936276 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1365/2726] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:15.989955 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a3de3-7cf6-4f28-a8c1-b32619fcb1b2 service placement] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:15.995984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180a033-d8d9-447b-8bab-7eb9eeb22500 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:15.996719 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180a033-d8d9-447b-8bab-7eb9eeb22500 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:15.997097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180a033-d8d9-447b-8bab-7eb9eeb22500 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:15.997456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180a033-d8d9-447b-8bab-7eb9eeb22500 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:15.999831 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f180a033-d8d9-447b-8bab-7eb9eeb22500 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:16.011324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180a033-d8d9-447b-8bab-7eb9eeb22500 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:16.012894 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f180a033-d8d9-447b-8bab-7eb9eeb22500 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:16.022369 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1362/2727] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:28:18.053606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd0afb-2514-4d64-9700-0abcc5ba039d service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:18.103453 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:18.118289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd0afb-2514-4d64-9700-0abcc5ba039d service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:18.122999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1693f8-8a8e-4124-9207-4a26761b6f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:18.123681 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1693f8-8a8e-4124-9207-4a26761b6f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:18.123966 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1693f8-8a8e-4124-9207-4a26761b6f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:18.124267 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1693f8-8a8e-4124-9207-4a26761b6f29 admin admin] PATH_INFO: `/v3/users/95c49b58a5fd4da682a88b27407d2a80` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:18.125496 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1693f8-8a8e-4124-9207-4a26761b6f29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c49b58a5fd4da682a88b27407d2a80)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:18.127456 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1693f8-8a8e-4124-9207-4a26761b6f29 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:18.182586 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cc1693f8-8a8e-4124-9207-4a26761b6f29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c49b58a5fd4da682a88b27407d2a80'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:18.183270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cc1693f8-8a8e-4124-9207-4a26761b6f29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c49b58a5fd4da682a88b27407d2a80'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:18.190401 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cc1693f8-8a8e-4124-9207-4a26761b6f29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c49b58a5fd4da682a88b27407d2a80'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:18.191550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cc1693f8-8a8e-4124-9207-4a26761b6f29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c49b58a5fd4da682a88b27407d2a80'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:18.207762 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-cc1693f8-8a8e-4124-9207-4a26761b6f29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c49b58a5fd4da682a88b27407d2a80'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:18.221597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1366/2728] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:28:18 2022] DELETE /identity/v3/users/95c49b58a5fd4da682a88b27407d2a80 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:18.226111 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cbe27-ff75-47fa-8335-3073e69032fa None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:18.226573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cbe27-ff75-47fa-8335-3073e69032fa None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:18.226708 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cbe27-ff75-47fa-8335-3073e69032fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:18.245198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7d4cbe27-ff75-47fa-8335-3073e69032fa None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:18.246379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7d4cbe27-ff75-47fa-8335-3073e69032fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:18.254957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=100 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:18.313028 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1363/2729] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:18.323441 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1693f8-8a8e-4124-9207-4a26761b6f29 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:18.325056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-cc1693f8-8a8e-4124-9207-4a26761b6f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:18.384391 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1693f8-8a8e-4124-9207-4a26761b6f29 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:18.388258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e18ee-176e-45f1-ae43-88543c83af75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:18.388908 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e18ee-176e-45f1-ae43-88543c83af75 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:18.389223 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e18ee-176e-45f1-ae43-88543c83af75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:18.389504 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e18ee-176e-45f1-ae43-88543c83af75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:18.399161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e18ee-176e-45f1-ae43-88543c83af75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:18.400722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e18ee-176e-45f1-ae43-88543c83af75 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:18.410643 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1367/2730] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:18.694911 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cbe27-ff75-47fa-8335-3073e69032fa None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:18.696404 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7d4cbe27-ff75-47fa-8335-3073e69032fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:18.710148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cbe27-ff75-47fa-8335-3073e69032fa None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:18.715393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:18.716071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:18.716353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:18.716631 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] PATH_INFO: `/v3/projects/50af00285d8543f1b1dbd82bf173aa0d` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:18.717805 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=50af00285d8543f1b1dbd82bf173aa0d)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:18.719909 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:18.766906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50af00285d8543f1b1dbd82bf173aa0d', 'request_id': 'req-7193cbd4-9d50-48bb-acca-fb3d26384bbe'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:18.773889 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50af00285d8543f1b1dbd82bf173aa0d', 'request_id': 'req-7193cbd4-9d50-48bb-acca-fb3d26384bbe'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:18.780941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50af00285d8543f1b1dbd82bf173aa0d', 'request_id': 'req-7193cbd4-9d50-48bb-acca-fb3d26384bbe'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:18.791396 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] The token cache is being invalidate because project 50af00285d8543f1b1dbd82bf173aa0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:28:18.791689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:18.792344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:18.793844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1364/2731] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:28:18 2022] DELETE /identity/v3/projects/50af00285d8543f1b1dbd82bf173aa0d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:18.798179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e18ee-176e-45f1-ae43-88543c83af75 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:18.808122 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e18ee-176e-45f1-ae43-88543c83af75 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:18.866048 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bca5b-dd33-48f5-9c8e-9468dfe671df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:18.866712 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bca5b-dd33-48f5-9c8e-9468dfe671df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:18.867010 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bca5b-dd33-48f5-9c8e-9468dfe671df admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:18.867292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bca5b-dd33-48f5-9c8e-9468dfe671df admin admin] PATH_INFO: `/v3/users/2af64dd44da14b51805fb0cc78a33c8d` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:18.868458 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bca5b-dd33-48f5-9c8e-9468dfe671df admin admin] RBAC: Authorizing `identity:delete_user(user_id=2af64dd44da14b51805fb0cc78a33c8d)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:18.870392 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bca5b-dd33-48f5-9c8e-9468dfe671df admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:19.006787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:19.042325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-331bca5b-dd33-48f5-9c8e-9468dfe671df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2af64dd44da14b51805fb0cc78a33c8d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.042983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-331bca5b-dd33-48f5-9c8e-9468dfe671df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2af64dd44da14b51805fb0cc78a33c8d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.049842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-331bca5b-dd33-48f5-9c8e-9468dfe671df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2af64dd44da14b51805fb0cc78a33c8d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.050472 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-331bca5b-dd33-48f5-9c8e-9468dfe671df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2af64dd44da14b51805fb0cc78a33c8d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.062184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-331bca5b-dd33-48f5-9c8e-9468dfe671df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2af64dd44da14b51805fb0cc78a33c8d'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.075392 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1368/2732] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:28:18 2022] DELETE /identity/v3/users/2af64dd44da14b51805fb0cc78a33c8d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:19.363213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:19.364698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:19.377670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193cbd4-9d50-48bb-acca-fb3d26384bbe admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:19.435203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:19.435841 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:19.436128 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:19.436397 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] PATH_INFO: `/v3/projects/827cf7b37159463aad4177b06560e2ae` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:19.437549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=827cf7b37159463aad4177b06560e2ae)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:19.439500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:19.480301 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:19.488477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '827cf7b37159463aad4177b06560e2ae', 'request_id': 'req-bb37cf91-a6c2-42a2-bace-55034e2f55e4'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.494497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '827cf7b37159463aad4177b06560e2ae', 'request_id': 'req-bb37cf91-a6c2-42a2-bace-55034e2f55e4'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.501174 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '827cf7b37159463aad4177b06560e2ae', 'request_id': 'req-bb37cf91-a6c2-42a2-bace-55034e2f55e4'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.512146 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] The token cache is being invalidate because project 827cf7b37159463aad4177b06560e2ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:28:19.512437 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.513064 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.514575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1365/2733] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:28:19 2022] DELETE /identity/v3/projects/827cf7b37159463aad4177b06560e2ae => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:19.518677 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bca5b-dd33-48f5-9c8e-9468dfe671df admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:19.520681 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-331bca5b-dd33-48f5-9c8e-9468dfe671df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:19.533413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bca5b-dd33-48f5-9c8e-9468dfe671df admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:19.594785 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25ab1e-156b-450a-a166-c8862ff813a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:19.595444 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25ab1e-156b-450a-a166-c8862ff813a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:19.595727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25ab1e-156b-450a-a166-c8862ff813a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:19.596002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25ab1e-156b-450a-a166-c8862ff813a3 admin admin] PATH_INFO: `/v3/users/506d254ce58c4fe68fc6a6a04f9821e2` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:19.597182 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25ab1e-156b-450a-a166-c8862ff813a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=506d254ce58c4fe68fc6a6a04f9821e2)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:19.599175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25ab1e-156b-450a-a166-c8862ff813a3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:19.648164 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2a25ab1e-156b-450a-a166-c8862ff813a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '506d254ce58c4fe68fc6a6a04f9821e2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.648806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2a25ab1e-156b-450a-a166-c8862ff813a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '506d254ce58c4fe68fc6a6a04f9821e2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.655766 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2a25ab1e-156b-450a-a166-c8862ff813a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '506d254ce58c4fe68fc6a6a04f9821e2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.656402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2a25ab1e-156b-450a-a166-c8862ff813a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '506d254ce58c4fe68fc6a6a04f9821e2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.668705 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2a25ab1e-156b-450a-a166-c8862ff813a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '506d254ce58c4fe68fc6a6a04f9821e2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:19.681795 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1369/2734] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:28:19 2022] DELETE /identity/v3/users/506d254ce58c4fe68fc6a6a04f9821e2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:19.958022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:19.959670 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:19.973038 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37cf91-a6c2-42a2-bace-55034e2f55e4 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:20.032664 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:20.033305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:20.033588 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:20.033865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734 admin admin] PATH_INFO: `/v3/projects/d6cf7d896f8441e199ff094bfa5b3c6b` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:20.035621 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6cf7d896f8441e199ff094bfa5b3c6b)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:20.037531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:20.084189 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6cf7d896f8441e199ff094bfa5b3c6b', 'request_id': 'req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:20.090143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6cf7d896f8441e199ff094bfa5b3c6b', 'request_id': 'req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:20.097297 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6cf7d896f8441e199ff094bfa5b3c6b', 'request_id': 'req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:20.108055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734 admin admin] The token cache is being invalidate because project d6cf7d896f8441e199ff094bfa5b3c6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:28:20.108387 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:20.109003 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-75ee8719-c7d5-4eed-9d85-a0c04d4fc734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:20.110479 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1366/2735] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:28:19 2022] DELETE /identity/v3/projects/d6cf7d896f8441e199ff094bfa5b3c6b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:20.116483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a9f05-c4f4-4fe3-97d6-e2b10801e679 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:20.116971 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a9f05-c4f4-4fe3-97d6-e2b10801e679 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:20.117092 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a9f05-c4f4-4fe3-97d6-e2b10801e679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:20.135900 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-d72a9f05-c4f4-4fe3-97d6-e2b10801e679 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:20.137086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-d72a9f05-c4f4-4fe3-97d6-e2b10801e679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:20.141222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:20.197511 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1370/2736] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:28:20.207196 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b378f-15c3-4e0b-b132-bd754caff591 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:20.207458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b378f-15c3-4e0b-b132-bd754caff591 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:20.207784 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b378f-15c3-4e0b-b132-bd754caff591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:20.228222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-085b378f-15c3-4e0b-b132-bd754caff591 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:20.229510 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-085b378f-15c3-4e0b-b132-bd754caff591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:20.237321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1367/2737] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:20.241736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a9f05-c4f4-4fe3-97d6-e2b10801e679 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:20.251602 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a9f05-c4f4-4fe3-97d6-e2b10801e679 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:20.256702 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be4942-7f32-4afc-a3fb-136ebed32e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:20.257307 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be4942-7f32-4afc-a3fb-136ebed32e3e admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:20.257590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be4942-7f32-4afc-a3fb-136ebed32e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:20.257865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be4942-7f32-4afc-a3fb-136ebed32e3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:20.258588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be4942-7f32-4afc-a3fb-136ebed32e3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:28:20.259512 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be4942-7f32-4afc-a3fb-136ebed32e3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:20.261116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be4942-7f32-4afc-a3fb-136ebed32e3e admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:20.281341 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1371/2738] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:28:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:28:20.286773 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b378f-15c3-4e0b-b132-bd754caff591 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:20.296074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b378f-15c3-4e0b-b132-bd754caff591 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:20.300799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94ca03-9be1-4fc5-b944-01cd13cd870a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:20.301406 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94ca03-9be1-4fc5-b944-01cd13cd870a admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:20.301718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94ca03-9be1-4fc5-b944-01cd13cd870a admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:20.301995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94ca03-9be1-4fc5-b944-01cd13cd870a admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:20.302737 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94ca03-9be1-4fc5-b944-01cd13cd870a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:20.304642 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94ca03-9be1-4fc5-b944-01cd13cd870a admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:20.331414 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1368/2739] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:28:20 2022] POST /identity/v3/projects => generated 372 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:28:20.336279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be4942-7f32-4afc-a3fb-136ebed32e3e admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:20.346312 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be4942-7f32-4afc-a3fb-136ebed32e3e admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:20.405150 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e7ee1-f78b-4e85-8a14-9a116151708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:20.405774 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e7ee1-f78b-4e85-8a14-9a116151708d admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:20.406244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e7ee1-f78b-4e85-8a14-9a116151708d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:20.406529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e7ee1-f78b-4e85-8a14-9a116151708d admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:20.407256 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e7ee1-f78b-4e85-8a14-9a116151708d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:20.409172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e7ee1-f78b-4e85-8a14-9a116151708d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:20.430727 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1372/2740] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:28:20 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:20.434880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94ca03-9be1-4fc5-b944-01cd13cd870a admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:20.446138 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94ca03-9be1-4fc5-b944-01cd13cd870a admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:20.451168 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599c71f-6c85-40a6-b0dc-57f735cfd11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:20.451791 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599c71f-6c85-40a6-b0dc-57f735cfd11b admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:20.452073 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599c71f-6c85-40a6-b0dc-57f735cfd11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:20.452346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599c71f-6c85-40a6-b0dc-57f735cfd11b admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:20.453095 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599c71f-6c85-40a6-b0dc-57f735cfd11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:20.454712 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e599c71f-6c85-40a6-b0dc-57f735cfd11b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:20.465781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1369/2741] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:20 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:20.470192 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e7ee1-f78b-4e85-8a14-9a116151708d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:20.480501 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e7ee1-f78b-4e85-8a14-9a116151708d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:20.485606 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29b2b7-aa8d-48d7-91a8-d034d186c08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:20.486227 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29b2b7-aa8d-48d7-91a8-d034d186c08f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:20.486509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29b2b7-aa8d-48d7-91a8-d034d186c08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:20.486818 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29b2b7-aa8d-48d7-91a8-d034d186c08f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:20.487516 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29b2b7-aa8d-48d7-91a8-d034d186c08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:20.489107 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29b2b7-aa8d-48d7-91a8-d034d186c08f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:20.499423 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1373/2742] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:20 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:28:20.504163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599c71f-6c85-40a6-b0dc-57f735cfd11b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:20.514439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599c71f-6c85-40a6-b0dc-57f735cfd11b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:20.519090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc71ca0-acb8-4445-b119-758fd6ae8729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:20.520359 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc71ca0-acb8-4445-b119-758fd6ae8729 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:20.520640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc71ca0-acb8-4445-b119-758fd6ae8729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:20.520915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc71ca0-acb8-4445-b119-758fd6ae8729 admin admin] PATH_INFO: `/v3/projects/70391619d2e94d3fb5f8814c0641d0b0/users/bc7181a7958540fca52dc006011e3f49/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:20.535245 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc71ca0-acb8-4445-b119-758fd6ae8729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70391619d2e94d3fb5f8814c0641d0b0, user_id=bc7181a7958540fca52dc006011e3f49, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:20.537887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc71ca0-acb8-4445-b119-758fd6ae8729 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:20.540186 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:20.555976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1370/2743] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:20 2022] PUT /identity/v3/projects/70391619d2e94d3fb5f8814c0641d0b0/users/bc7181a7958540fca52dc006011e3f49/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:20.561183 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e7c7d-acdf-46d0-8ee1-db04bfd62998 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:20.561471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e7c7d-acdf-46d0-8ee1-db04bfd62998 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:20.561784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e7c7d-acdf-46d0-8ee1-db04bfd62998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:20.580979 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-804e7c7d-acdf-46d0-8ee1-db04bfd62998 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:20.582836 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-804e7c7d-acdf-46d0-8ee1-db04bfd62998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:20.640245 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1374/2744] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:20.649742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc71ca0-acb8-4445-b119-758fd6ae8729 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:20.651439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-0dc71ca0-acb8-4445-b119-758fd6ae8729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:20.709652 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc71ca0-acb8-4445-b119-758fd6ae8729 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:20.713259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17579690-9294-4b27-aefd-965cde900b60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:20.713865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17579690-9294-4b27-aefd-965cde900b60 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:20.714506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17579690-9294-4b27-aefd-965cde900b60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:20.714817 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17579690-9294-4b27-aefd-965cde900b60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:20.724777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17579690-9294-4b27-aefd-965cde900b60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:20.726197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17579690-9294-4b27-aefd-965cde900b60 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:20.737950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1371/2745] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:21.348481 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aaebb5-36f5-408d-9248-01e76be34e17 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:21.349015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aaebb5-36f5-408d-9248-01e76be34e17 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:21.349015 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aaebb5-36f5-408d-9248-01e76be34e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:21.377164 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-69aaebb5-36f5-408d-9248-01e76be34e17 None None] MFA Rules not processed for user `bc7181a7958540fca52dc006011e3f49`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:21.378381 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-69aaebb5-36f5-408d-9248-01e76be34e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:21.384771 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:21.450101 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1375/2746] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:21.455053 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17579690-9294-4b27-aefd-965cde900b60 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:21.465142 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17579690-9294-4b27-aefd-965cde900b60 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:21.469785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4709d6c-dc62-4a4c-8ca4-bd941da525d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:21.470386 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4709d6c-dc62-4a4c-8ca4-bd941da525d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:21.470683 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4709d6c-dc62-4a4c-8ca4-bd941da525d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:21.470956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4709d6c-dc62-4a4c-8ca4-bd941da525d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:21.471619 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4709d6c-dc62-4a4c-8ca4-bd941da525d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:21.474052 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4709d6c-dc62-4a4c-8ca4-bd941da525d8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:21.502192 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1372/2747] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:28:21 2022] POST /identity/v3/projects => generated 374 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:21.506346 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aaebb5-36f5-408d-9248-01e76be34e17 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:21.517070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aaebb5-36f5-408d-9248-01e76be34e17 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:21.573668 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f8610-39ba-4814-87da-1e655d071945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:21.574275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f8610-39ba-4814-87da-1e655d071945 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:21.574557 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f8610-39ba-4814-87da-1e655d071945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:21.574852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f8610-39ba-4814-87da-1e655d071945 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:21.575849 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f8610-39ba-4814-87da-1e655d071945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:21.577736 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f8610-39ba-4814-87da-1e655d071945 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:21.600799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1376/2748] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:28:21 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:21.604776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4709d6c-dc62-4a4c-8ca4-bd941da525d8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:21.615071 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4709d6c-dc62-4a4c-8ca4-bd941da525d8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:21.619626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cbca0-4cdd-450b-9421-6dd7c2ed6954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:21.620228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cbca0-4cdd-450b-9421-6dd7c2ed6954 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:21.620513 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cbca0-4cdd-450b-9421-6dd7c2ed6954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:21.620789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cbca0-4cdd-450b-9421-6dd7c2ed6954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:21.621478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cbca0-4cdd-450b-9421-6dd7c2ed6954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:21.623151 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cbca0-4cdd-450b-9421-6dd7c2ed6954 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:21.633488 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1373/2749] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:21 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:21.637509 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f8610-39ba-4814-87da-1e655d071945 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:21.647681 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f8610-39ba-4814-87da-1e655d071945 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:21.652172 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc1ac4-d4e8-4bad-b2ab-9f4640a0afe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:21.652817 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc1ac4-d4e8-4bad-b2ab-9f4640a0afe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:21.653093 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc1ac4-d4e8-4bad-b2ab-9f4640a0afe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:21.653428 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc1ac4-d4e8-4bad-b2ab-9f4640a0afe0 admin admin] PATH_INFO: `/v3/projects/8b544313afdb477da3133c142254ba37/users/2b99419804e04c748a686f45cde7c78f/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:21.668019 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc1ac4-d4e8-4bad-b2ab-9f4640a0afe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b544313afdb477da3133c142254ba37, user_id=2b99419804e04c748a686f45cde7c78f, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:21.670779 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc1ac4-d4e8-4bad-b2ab-9f4640a0afe0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:21.683240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1377/2750] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:21 2022] PUT /identity/v3/projects/8b544313afdb477da3133c142254ba37/users/2b99419804e04c748a686f45cde7c78f/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:21.687324 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cbca0-4cdd-450b-9421-6dd7c2ed6954 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:21.697345 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cbca0-4cdd-450b-9421-6dd7c2ed6954 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:21.753927 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780eef4-2a1c-429b-8376-515ed4969b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:21.754605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780eef4-2a1c-429b-8376-515ed4969b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:21.754988 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780eef4-2a1c-429b-8376-515ed4969b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:21.755265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780eef4-2a1c-429b-8376-515ed4969b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:21.755961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780eef4-2a1c-429b-8376-515ed4969b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:21.757569 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780eef4-2a1c-429b-8376-515ed4969b6e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:21.767596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1374/2751] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:21 2022] GET /identity/v3/roles => generated 1427 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:21.772505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc1ac4-d4e8-4bad-b2ab-9f4640a0afe0 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:21.782847 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc1ac4-d4e8-4bad-b2ab-9f4640a0afe0 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:21.788804 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac1698-2d07-477a-867a-9a84bb2a84aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:21.789593 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac1698-2d07-477a-867a-9a84bb2a84aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:21.789955 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac1698-2d07-477a-867a-9a84bb2a84aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:21.790340 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac1698-2d07-477a-867a-9a84bb2a84aa admin admin] PATH_INFO: `/v3/projects/8b544313afdb477da3133c142254ba37/users/2b99419804e04c748a686f45cde7c78f/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:21.792997 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac1698-2d07-477a-867a-9a84bb2a84aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b544313afdb477da3133c142254ba37, user_id=2b99419804e04c748a686f45cde7c78f, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:21.797220 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac1698-2d07-477a-867a-9a84bb2a84aa admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:21.808247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1378/2752] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:21 2022] PUT /identity/v3/projects/8b544313afdb477da3133c142254ba37/users/2b99419804e04c748a686f45cde7c78f/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:22.606069 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be80fba-bd43-48c3-8d42-2214c0330355 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:22.606589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be80fba-bd43-48c3-8d42-2214c0330355 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:22.606589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be80fba-bd43-48c3-8d42-2214c0330355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:22.635722 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-4be80fba-bd43-48c3-8d42-2214c0330355 None None] MFA Rules not processed for user `2b99419804e04c748a686f45cde7c78f`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:22.636949 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4be80fba-bd43-48c3-8d42-2214c0330355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:22.686599 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:22.711391 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1375/2753] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:22.722956 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac1698-2d07-477a-867a-9a84bb2a84aa admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:22.724846 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-36ac1698-2d07-477a-867a-9a84bb2a84aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:22.782730 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac1698-2d07-477a-867a-9a84bb2a84aa admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:22.786291 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d26183-66ea-4772-ab0a-a325839c133c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:22.787021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d26183-66ea-4772-ab0a-a325839c133c service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:22.787320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d26183-66ea-4772-ab0a-a325839c133c service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:22.787573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d26183-66ea-4772-ab0a-a325839c133c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:22.797037 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:22.847168 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d26183-66ea-4772-ab0a-a325839c133c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:22.848605 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d26183-66ea-4772-ab0a-a325839c133c service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:22.857334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1379/2754] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:24.211844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be80fba-bd43-48c3-8d42-2214c0330355 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:24.213228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4be80fba-bd43-48c3-8d42-2214c0330355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:24.225379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be80fba-bd43-48c3-8d42-2214c0330355 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:24.228886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80807ed-87a2-4df5-a21d-382d68b5be9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:24.229480 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80807ed-87a2-4df5-a21d-382d68b5be9e service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:24.229758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80807ed-87a2-4df5-a21d-382d68b5be9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:24.230030 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80807ed-87a2-4df5-a21d-382d68b5be9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:24.239859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80807ed-87a2-4df5-a21d-382d68b5be9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:24.241267 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80807ed-87a2-4df5-a21d-382d68b5be9e service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:24.250675 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1376/2755] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:32.853354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=22 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:34.246489 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=13 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:36.714124 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d26183-66ea-4772-ab0a-a325839c133c service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:36.715607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b0d26183-66ea-4772-ab0a-a325839c133c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:36.791027 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d26183-66ea-4772-ab0a-a325839c133c service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:36.796362 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5e5fb-ab40-4be2-80e2-818baea755a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:36.796981 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5e5fb-ab40-4be2-80e2-818baea755a9 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:36.797261 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5e5fb-ab40-4be2-80e2-818baea755a9 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:36.797528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5e5fb-ab40-4be2-80e2-818baea755a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:36.799291 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bbd5e5fb-ab40-4be2-80e2-818baea755a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:36.811190 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5e5fb-ab40-4be2-80e2-818baea755a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:36.812798 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5e5fb-ab40-4be2-80e2-818baea755a9 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:36.823626 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1380/2756] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:37.552118 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80807ed-87a2-4df5-a21d-382d68b5be9e service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:37.614783 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80807ed-87a2-4df5-a21d-382d68b5be9e service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:37.619442 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6ed8a-03a1-44ac-a6c5-05f71ce89eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:37.620084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6ed8a-03a1-44ac-a6c5-05f71ce89eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:37.620368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6ed8a-03a1-44ac-a6c5-05f71ce89eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:37.620647 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6ed8a-03a1-44ac-a6c5-05f71ce89eb5 admin admin] PATH_INFO: `/v3/users/bc7181a7958540fca52dc006011e3f49` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:37.621811 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6ed8a-03a1-44ac-a6c5-05f71ce89eb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc7181a7958540fca52dc006011e3f49)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:37.623772 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6ed8a-03a1-44ac-a6c5-05f71ce89eb5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:37.669409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6af6ed8a-03a1-44ac-a6c5-05f71ce89eb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc7181a7958540fca52dc006011e3f49'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:37.676910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6af6ed8a-03a1-44ac-a6c5-05f71ce89eb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc7181a7958540fca52dc006011e3f49'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:37.677843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6af6ed8a-03a1-44ac-a6c5-05f71ce89eb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc7181a7958540fca52dc006011e3f49'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:37.678582 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6af6ed8a-03a1-44ac-a6c5-05f71ce89eb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc7181a7958540fca52dc006011e3f49'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:37.692849 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-6af6ed8a-03a1-44ac-a6c5-05f71ce89eb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc7181a7958540fca52dc006011e3f49'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:37.708671 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1377/2757] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:28:37 2022] DELETE /identity/v3/users/bc7181a7958540fca52dc006011e3f49 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:37.873548 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5e5fb-ab40-4be2-80e2-818baea755a9 service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:37.884715 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5e5fb-ab40-4be2-80e2-818baea755a9 service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:37.940530 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:37.941677 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:37.941960 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:37.942235 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] PATH_INFO: `/v3/projects/70391619d2e94d3fb5f8814c0641d0b0` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:37.943477 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70391619d2e94d3fb5f8814c0641d0b0)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:37.946997 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:37.995299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70391619d2e94d3fb5f8814c0641d0b0', 'request_id': 'req-bcf5a726-8d03-479c-90f8-d09232eea784'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.001589 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70391619d2e94d3fb5f8814c0641d0b0', 'request_id': 'req-bcf5a726-8d03-479c-90f8-d09232eea784'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.012816 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70391619d2e94d3fb5f8814c0641d0b0', 'request_id': 'req-bcf5a726-8d03-479c-90f8-d09232eea784'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.025872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] The token cache is being invalidate because project 70391619d2e94d3fb5f8814c0641d0b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:28:38.026165 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.026826 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.028406 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1381/2758] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:28:37 2022] DELETE /identity/v3/projects/70391619d2e94d3fb5f8814c0641d0b0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:38.032478 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6ed8a-03a1-44ac-a6c5-05f71ce89eb5 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:38.033918 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-6af6ed8a-03a1-44ac-a6c5-05f71ce89eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:38.047181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6ed8a-03a1-44ac-a6c5-05f71ce89eb5 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:38.102596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc796d93-e0c2-4bf0-8ff2-219cb1743462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:38.103254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc796d93-e0c2-4bf0-8ff2-219cb1743462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:38.103537 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc796d93-e0c2-4bf0-8ff2-219cb1743462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:38.103813 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc796d93-e0c2-4bf0-8ff2-219cb1743462 admin admin] PATH_INFO: `/v3/users/2b99419804e04c748a686f45cde7c78f` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:38.104998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc796d93-e0c2-4bf0-8ff2-219cb1743462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b99419804e04c748a686f45cde7c78f)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:38.106936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc796d93-e0c2-4bf0-8ff2-219cb1743462 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:38.128822 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:38.153067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dc796d93-e0c2-4bf0-8ff2-219cb1743462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b99419804e04c748a686f45cde7c78f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.161190 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dc796d93-e0c2-4bf0-8ff2-219cb1743462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b99419804e04c748a686f45cde7c78f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.161970 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dc796d93-e0c2-4bf0-8ff2-219cb1743462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b99419804e04c748a686f45cde7c78f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.162759 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dc796d93-e0c2-4bf0-8ff2-219cb1743462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b99419804e04c748a686f45cde7c78f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.176254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-dc796d93-e0c2-4bf0-8ff2-219cb1743462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b99419804e04c748a686f45cde7c78f'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.188328 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1378/2759] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:28:38 2022] DELETE /identity/v3/users/2b99419804e04c748a686f45cde7c78f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:38.365479 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:38.367200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:38.383551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5a726-8d03-479c-90f8-d09232eea784 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:38.403026 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:38.448417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceac914-d405-4423-9e31-533b95ca24c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:38.449059 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceac914-d405-4423-9e31-533b95ca24c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:38.449337 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceac914-d405-4423-9e31-533b95ca24c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:38.449608 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceac914-d405-4423-9e31-533b95ca24c2 admin admin] PATH_INFO: `/v3/projects/8b544313afdb477da3133c142254ba37` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:38.451069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceac914-d405-4423-9e31-533b95ca24c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b544313afdb477da3133c142254ba37)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:38.452996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceac914-d405-4423-9e31-533b95ca24c2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:38.500603 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ceac914-d405-4423-9e31-533b95ca24c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b544313afdb477da3133c142254ba37', 'request_id': 'req-5ceac914-d405-4423-9e31-533b95ca24c2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.507115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ceac914-d405-4423-9e31-533b95ca24c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b544313afdb477da3133c142254ba37', 'request_id': 'req-5ceac914-d405-4423-9e31-533b95ca24c2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.513120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ceac914-d405-4423-9e31-533b95ca24c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b544313afdb477da3133c142254ba37', 'request_id': 'req-5ceac914-d405-4423-9e31-533b95ca24c2'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.525926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ceac914-d405-4423-9e31-533b95ca24c2 admin admin] The token cache is being invalidate because project 8b544313afdb477da3133c142254ba37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:28:38.526221 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ceac914-d405-4423-9e31-533b95ca24c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.527187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-5ceac914-d405-4423-9e31-533b95ca24c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:38.528695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1382/2760] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:28:38 2022] DELETE /identity/v3/projects/8b544313afdb477da3133c142254ba37 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:38.533806 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1983d7-7f5c-40f6-b5f9-57743f5cd5f3 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:38.534596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1983d7-7f5c-40f6-b5f9-57743f5cd5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:38.534915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1983d7-7f5c-40f6-b5f9-57743f5cd5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:38.556957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7f1983d7-7f5c-40f6-b5f9-57743f5cd5f3 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:38.558178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7f1983d7-7f5c-40f6-b5f9-57743f5cd5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:38.620630 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1379/2761] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:38.628830 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8556c64-d2ba-47bf-ad99-f342844d128a None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:38.629102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8556c64-d2ba-47bf-ad99-f342844d128a None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:38.629518 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8556c64-d2ba-47bf-ad99-f342844d128a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:38.650219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-e8556c64-d2ba-47bf-ad99-f342844d128a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:38.651484 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-e8556c64-d2ba-47bf-ad99-f342844d128a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:38.660259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1383/2762] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:38.664279 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1983d7-7f5c-40f6-b5f9-57743f5cd5f3 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:38.674288 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1983d7-7f5c-40f6-b5f9-57743f5cd5f3 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:38.679034 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de65485-f014-4890-944c-5b14aef02531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:38.679649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de65485-f014-4890-944c-5b14aef02531 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:38.679930 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de65485-f014-4890-944c-5b14aef02531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:38.680205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de65485-f014-4890-944c-5b14aef02531 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:38.680906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de65485-f014-4890-944c-5b14aef02531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:28:38.681214 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de65485-f014-4890-944c-5b14aef02531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:38.682839 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de65485-f014-4890-944c-5b14aef02531 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:38.695654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1380/2763] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:28:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:28:38.699973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8556c64-d2ba-47bf-ad99-f342844d128a None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:38.710099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8556c64-d2ba-47bf-ad99-f342844d128a None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:38.714700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ddd62-879e-4a9f-b2e6-b01aa49a54ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:38.715331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ddd62-879e-4a9f-b2e6-b01aa49a54ce admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:38.715612 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ddd62-879e-4a9f-b2e6-b01aa49a54ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:38.715888 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ddd62-879e-4a9f-b2e6-b01aa49a54ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:38.716561 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ddd62-879e-4a9f-b2e6-b01aa49a54ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:38.718521 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ddd62-879e-4a9f-b2e6-b01aa49a54ce admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:38.746016 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1384/2764] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:28:38 2022] POST /identity/v3/projects => generated 386 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:38.750844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de65485-f014-4890-944c-5b14aef02531 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:38.760945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de65485-f014-4890-944c-5b14aef02531 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:38.818129 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76564afe-a492-493d-a205-a33edb786eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:38.818859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76564afe-a492-493d-a205-a33edb786eee admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:38.819150 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76564afe-a492-493d-a205-a33edb786eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:38.819431 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76564afe-a492-493d-a205-a33edb786eee admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:38.820103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76564afe-a492-493d-a205-a33edb786eee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:38.822024 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76564afe-a492-493d-a205-a33edb786eee admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:38.843687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1381/2765] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:28:38 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:28:38.848412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ddd62-879e-4a9f-b2e6-b01aa49a54ce admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:38.858686 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ddd62-879e-4a9f-b2e6-b01aa49a54ce admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:38.863255 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0600ca-76ca-457b-8d43-949fd2dc0e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:38.863861 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0600ca-76ca-457b-8d43-949fd2dc0e3f admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:38.864139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0600ca-76ca-457b-8d43-949fd2dc0e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:38.864426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0600ca-76ca-457b-8d43-949fd2dc0e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:38.865094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0600ca-76ca-457b-8d43-949fd2dc0e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:38.866700 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0600ca-76ca-457b-8d43-949fd2dc0e3f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:38.877157 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1385/2766] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:38 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:38.881105 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76564afe-a492-493d-a205-a33edb786eee admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:38.891529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76564afe-a492-493d-a205-a33edb786eee admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:38.896810 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1dda4-7cd4-4de9-a3e3-5aa099c3a303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:38.897437 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1dda4-7cd4-4de9-a3e3-5aa099c3a303 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:38.897721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1dda4-7cd4-4de9-a3e3-5aa099c3a303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:38.898006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1dda4-7cd4-4de9-a3e3-5aa099c3a303 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:38.898906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1dda4-7cd4-4de9-a3e3-5aa099c3a303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:38.900524 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1dda4-7cd4-4de9-a3e3-5aa099c3a303 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:38.915979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1382/2767] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:38 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:38.920292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0600ca-76ca-457b-8d43-949fd2dc0e3f admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:38.930409 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0600ca-76ca-457b-8d43-949fd2dc0e3f admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:38.935194 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc57f4-1bdc-4808-b3b7-62cf92a7e89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:38.935850 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc57f4-1bdc-4808-b3b7-62cf92a7e89d admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:38.936134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc57f4-1bdc-4808-b3b7-62cf92a7e89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:38.936412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc57f4-1bdc-4808-b3b7-62cf92a7e89d admin admin] PATH_INFO: `/v3/projects/0e857003c0b040e49dc03c51c36ffe43/users/52e2b800a5284c54a0f372f337fede0b/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:38.952091 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc57f4-1bdc-4808-b3b7-62cf92a7e89d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e857003c0b040e49dc03c51c36ffe43, user_id=52e2b800a5284c54a0f372f337fede0b, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:38.954747 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc57f4-1bdc-4808-b3b7-62cf92a7e89d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:38.967885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1386/2768] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:38 2022] PUT /identity/v3/projects/0e857003c0b040e49dc03c51c36ffe43/users/52e2b800a5284c54a0f372f337fede0b/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:38.980020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ab8a9-7044-4168-8394-aba2a783b404 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:38.980308 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ab8a9-7044-4168-8394-aba2a783b404 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:38.980585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ab8a9-7044-4168-8394-aba2a783b404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:38.988905 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:39.016532 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-5c2ab8a9-7044-4168-8394-aba2a783b404 None None] MFA Rules not processed for user `52e2b800a5284c54a0f372f337fede0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:39.017762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-5c2ab8a9-7044-4168-8394-aba2a783b404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:39.081693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1383/2769] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:39.093044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc57f4-1bdc-4808-b3b7-62cf92a7e89d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:39.094786 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-02dc57f4-1bdc-4808-b3b7-62cf92a7e89d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:39.145905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:39.152293 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc57f4-1bdc-4808-b3b7-62cf92a7e89d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:39.157006 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056e0fc-6ea5-4778-b925-2fc2a79d3e74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:39.157615 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056e0fc-6ea5-4778-b925-2fc2a79d3e74 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:39.157897 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056e0fc-6ea5-4778-b925-2fc2a79d3e74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:39.158200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056e0fc-6ea5-4778-b925-2fc2a79d3e74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:39.166476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056e0fc-6ea5-4778-b925-2fc2a79d3e74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:39.167933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056e0fc-6ea5-4778-b925-2fc2a79d3e74 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:39.177212 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1387/2770] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:41.727198 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ab8a9-7044-4168-8394-aba2a783b404 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:41.788945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ab8a9-7044-4168-8394-aba2a783b404 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:41.793467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e9779-5c86-46f2-b83e-cb186969f3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:41.794095 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e9779-5c86-46f2-b83e-cb186969f3be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:41.794376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e9779-5c86-46f2-b83e-cb186969f3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:41.794649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e9779-5c86-46f2-b83e-cb186969f3be admin admin] PATH_INFO: `/v3/users/52e2b800a5284c54a0f372f337fede0b` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:41.795882 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e9779-5c86-46f2-b83e-cb186969f3be admin admin] RBAC: Authorizing `identity:delete_user(user_id=52e2b800a5284c54a0f372f337fede0b)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:41.797849 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3e9779-5c86-46f2-b83e-cb186969f3be admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:41.846227 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c3e9779-5c86-46f2-b83e-cb186969f3be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52e2b800a5284c54a0f372f337fede0b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:41.853098 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c3e9779-5c86-46f2-b83e-cb186969f3be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52e2b800a5284c54a0f372f337fede0b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:41.853835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c3e9779-5c86-46f2-b83e-cb186969f3be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52e2b800a5284c54a0f372f337fede0b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:41.854459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c3e9779-5c86-46f2-b83e-cb186969f3be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52e2b800a5284c54a0f372f337fede0b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:41.866855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1c3e9779-5c86-46f2-b83e-cb186969f3be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52e2b800a5284c54a0f372f337fede0b'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:41.880074 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1384/2771] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:28:41 2022] DELETE /identity/v3/users/52e2b800a5284c54a0f372f337fede0b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:41.884539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206b1ce-299b-47d6-99c2-dafe2208505b None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:41.885008 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206b1ce-299b-47d6-99c2-dafe2208505b None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:41.885127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206b1ce-299b-47d6-99c2-dafe2208505b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:41.903823 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7206b1ce-299b-47d6-99c2-dafe2208505b None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:41.905018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7206b1ce-299b-47d6-99c2-dafe2208505b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:41.962044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1388/2772] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:41.972184 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e9779-5c86-46f2-b83e-cb186969f3be admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:41.973589 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1c3e9779-5c86-46f2-b83e-cb186969f3be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:41.988601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:42.032376 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e9779-5c86-46f2-b83e-cb186969f3be admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:42.036479 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e21a1-2b4a-4ace-afca-00d30f6b6df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:42.037085 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e21a1-2b4a-4ace-afca-00d30f6b6df6 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:42.037364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e21a1-2b4a-4ace-afca-00d30f6b6df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:42.037635 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e21a1-2b4a-4ace-afca-00d30f6b6df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:42.046471 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e21a1-2b4a-4ace-afca-00d30f6b6df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:42.047982 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e21a1-2b4a-4ace-afca-00d30f6b6df6 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:42.057887 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1385/2773] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:42.332437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206b1ce-299b-47d6-99c2-dafe2208505b None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:42.333880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7206b1ce-299b-47d6-99c2-dafe2208505b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:42.345637 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206b1ce-299b-47d6-99c2-dafe2208505b None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:42.350811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03120f9-a1de-424a-9ffd-bda3f9b38a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:42.351458 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03120f9-a1de-424a-9ffd-bda3f9b38a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:42.351812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03120f9-a1de-424a-9ffd-bda3f9b38a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:42.352003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03120f9-a1de-424a-9ffd-bda3f9b38a70 admin admin] PATH_INFO: `/v3/projects/0e857003c0b040e49dc03c51c36ffe43` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:42.353179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03120f9-a1de-424a-9ffd-bda3f9b38a70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e857003c0b040e49dc03c51c36ffe43)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:42.355103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03120f9-a1de-424a-9ffd-bda3f9b38a70 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:42.410475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d03120f9-a1de-424a-9ffd-bda3f9b38a70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e857003c0b040e49dc03c51c36ffe43', 'request_id': 'req-d03120f9-a1de-424a-9ffd-bda3f9b38a70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:42.417167 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d03120f9-a1de-424a-9ffd-bda3f9b38a70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e857003c0b040e49dc03c51c36ffe43', 'request_id': 'req-d03120f9-a1de-424a-9ffd-bda3f9b38a70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:42.423213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d03120f9-a1de-424a-9ffd-bda3f9b38a70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e857003c0b040e49dc03c51c36ffe43', 'request_id': 'req-d03120f9-a1de-424a-9ffd-bda3f9b38a70'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:42.437476 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d03120f9-a1de-424a-9ffd-bda3f9b38a70 admin admin] The token cache is being invalidate because project 0e857003c0b040e49dc03c51c36ffe43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:28:42.437789 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d03120f9-a1de-424a-9ffd-bda3f9b38a70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:42.438383 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d03120f9-a1de-424a-9ffd-bda3f9b38a70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:42.439914 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1389/2774] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:28:42 2022] DELETE /identity/v3/projects/0e857003c0b040e49dc03c51c36ffe43 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:42.444923 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a00ad-5c01-46d9-ae60-041266f86e73 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:42.445213 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a00ad-5c01-46d9-ae60-041266f86e73 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:42.445549 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a00ad-5c01-46d9-ae60-041266f86e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:42.467103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-cf2a00ad-5c01-46d9-ae60-041266f86e73 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:42.469029 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-cf2a00ad-5c01-46d9-ae60-041266f86e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:42.529953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1386/2775] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:28:42.538219 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9b6b-b764-4cd1-8bfa-c191d75a3314 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:42.538504 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9b6b-b764-4cd1-8bfa-c191d75a3314 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:42.538823 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f9b6b-b764-4cd1-8bfa-c191d75a3314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:42.561287 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9e1f9b6b-b764-4cd1-8bfa-c191d75a3314 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:42.562492 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9e1f9b6b-b764-4cd1-8bfa-c191d75a3314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:42.570908 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1390/2776] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:42.575043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a00ad-5c01-46d9-ae60-041266f86e73 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:42.585864 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a00ad-5c01-46d9-ae60-041266f86e73 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:42.590578 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6dad4-54a4-41ec-9748-b7f1a05eacb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:42.591283 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6dad4-54a4-41ec-9748-b7f1a05eacb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:42.591641 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6dad4-54a4-41ec-9748-b7f1a05eacb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:42.591991 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6dad4-54a4-41ec-9748-b7f1a05eacb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:42.592729 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6dad4-54a4-41ec-9748-b7f1a05eacb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:28:42.593048 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6dad4-54a4-41ec-9748-b7f1a05eacb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:42.594764 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6dad4-54a4-41ec-9748-b7f1a05eacb0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:42.607814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1387/2777] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:28:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:28:42.612289 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f9b6b-b764-4cd1-8bfa-c191d75a3314 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:42.621677 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f9b6b-b764-4cd1-8bfa-c191d75a3314 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:42.627222 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f345c-c3fb-41f6-9f88-b6dcb68f8579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:42.627866 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f345c-c3fb-41f6-9f88-b6dcb68f8579 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:42.628147 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f345c-c3fb-41f6-9f88-b6dcb68f8579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:42.628523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f345c-c3fb-41f6-9f88-b6dcb68f8579 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:42.629202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f345c-c3fb-41f6-9f88-b6dcb68f8579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:42.631186 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f345c-c3fb-41f6-9f88-b6dcb68f8579 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:42.660128 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1391/2778] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:28:42 2022] POST /identity/v3/projects => generated 376 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:28:42.664340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6dad4-54a4-41ec-9748-b7f1a05eacb0 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:42.674246 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6dad4-54a4-41ec-9748-b7f1a05eacb0 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:42.730813 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7a389-1b7f-4c99-8ba4-e64136abe4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:42.731521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7a389-1b7f-4c99-8ba4-e64136abe4bd admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:42.731693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7a389-1b7f-4c99-8ba4-e64136abe4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:42.731971 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7a389-1b7f-4c99-8ba4-e64136abe4bd admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:42.732633 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7a389-1b7f-4c99-8ba4-e64136abe4bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:42.734555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7a389-1b7f-4c99-8ba4-e64136abe4bd admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:42.755084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1388/2779] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:28:42 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:28:42.759231 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f345c-c3fb-41f6-9f88-b6dcb68f8579 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:42.770244 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f345c-c3fb-41f6-9f88-b6dcb68f8579 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:42.775007 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8800bf-5c53-47ac-b57a-41eac0b194ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:42.775608 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8800bf-5c53-47ac-b57a-41eac0b194ba admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:42.775913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8800bf-5c53-47ac-b57a-41eac0b194ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:42.776184 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8800bf-5c53-47ac-b57a-41eac0b194ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:42.776873 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8800bf-5c53-47ac-b57a-41eac0b194ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:42.778440 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8800bf-5c53-47ac-b57a-41eac0b194ba admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:42.788363 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1392/2780] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:42 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:42.792193 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7a389-1b7f-4c99-8ba4-e64136abe4bd admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:42.802161 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7a389-1b7f-4c99-8ba4-e64136abe4bd admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:42.806852 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf8a22-37f4-4315-a609-393913414e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:42.807460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf8a22-37f4-4315-a609-393913414e5d admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:42.807744 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf8a22-37f4-4315-a609-393913414e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:42.808022 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf8a22-37f4-4315-a609-393913414e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:42.808720 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf8a22-37f4-4315-a609-393913414e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:42.810302 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf8a22-37f4-4315-a609-393913414e5d admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:42.821526 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1389/2781] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:42 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:42.826331 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8800bf-5c53-47ac-b57a-41eac0b194ba admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:42.836824 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8800bf-5c53-47ac-b57a-41eac0b194ba admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:42.841392 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f45465-2227-41ff-a76a-a2d86771105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:42.842044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f45465-2227-41ff-a76a-a2d86771105d admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:42.842321 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f45465-2227-41ff-a76a-a2d86771105d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:42.842594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f45465-2227-41ff-a76a-a2d86771105d admin admin] PATH_INFO: `/v3/projects/d767fdb835ce4bcf937bb0e17aa5bf68/users/aab7f520be794ec0b977b319c8af66ed/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:42.856915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f45465-2227-41ff-a76a-a2d86771105d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d767fdb835ce4bcf937bb0e17aa5bf68, user_id=aab7f520be794ec0b977b319c8af66ed, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:42.859584 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f45465-2227-41ff-a76a-a2d86771105d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:42.872501 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1393/2782] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:42 2022] PUT /identity/v3/projects/d767fdb835ce4bcf937bb0e17aa5bf68/users/aab7f520be794ec0b977b319c8af66ed/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:42.884534 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0afd82-eb8a-4e8f-a41a-0077024c9e76 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:42.884824 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0afd82-eb8a-4e8f-a41a-0077024c9e76 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:42.885103 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0afd82-eb8a-4e8f-a41a-0077024c9e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:42.913232 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ec0afd82-eb8a-4e8f-a41a-0077024c9e76 None None] MFA Rules not processed for user `aab7f520be794ec0b977b319c8af66ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:42.914458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ec0afd82-eb8a-4e8f-a41a-0077024c9e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:42.923538 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:42.979662 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1390/2783] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:42.990173 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f45465-2227-41ff-a76a-a2d86771105d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:42.992004 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-93f45465-2227-41ff-a76a-a2d86771105d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:43.003425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:43.052303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f45465-2227-41ff-a76a-a2d86771105d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:43.055948 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb20194-410f-4e08-8bfa-503380d9c0a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:43.056562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb20194-410f-4e08-8bfa-503380d9c0a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:43.056843 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb20194-410f-4e08-8bfa-503380d9c0a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:43.057115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb20194-410f-4e08-8bfa-503380d9c0a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:43.065870 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb20194-410f-4e08-8bfa-503380d9c0a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:43.067304 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb20194-410f-4e08-8bfa-503380d9c0a3 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:43.076145 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1394/2784] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:52.908131 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0afd82-eb8a-4e8f-a41a-0077024c9e76 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:52.970871 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0afd82-eb8a-4e8f-a41a-0077024c9e76 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:52.975377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5fda7-716b-4cf2-bba8-848fc64e6c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:52.976004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5fda7-716b-4cf2-bba8-848fc64e6c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:52.976283 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5fda7-716b-4cf2-bba8-848fc64e6c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:52.976555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5fda7-716b-4cf2-bba8-848fc64e6c8b admin admin] PATH_INFO: `/v3/users/aab7f520be794ec0b977b319c8af66ed` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:52.977700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5fda7-716b-4cf2-bba8-848fc64e6c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab7f520be794ec0b977b319c8af66ed)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:52.979707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5fda7-716b-4cf2-bba8-848fc64e6c8b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:53.025925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f7a5fda7-716b-4cf2-bba8-848fc64e6c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab7f520be794ec0b977b319c8af66ed'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:53.033287 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f7a5fda7-716b-4cf2-bba8-848fc64e6c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab7f520be794ec0b977b319c8af66ed'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:53.033908 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f7a5fda7-716b-4cf2-bba8-848fc64e6c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab7f520be794ec0b977b319c8af66ed'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:53.034552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f7a5fda7-716b-4cf2-bba8-848fc64e6c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab7f520be794ec0b977b319c8af66ed'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:53.046696 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f7a5fda7-716b-4cf2-bba8-848fc64e6c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab7f520be794ec0b977b319c8af66ed'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:53.059360 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1391/2785] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:28:52 2022] DELETE /identity/v3/users/aab7f520be794ec0b977b319c8af66ed => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:53.063820 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a038767-a3ca-4159-9b9a-acf58d5bfe79 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:53.064310 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a038767-a3ca-4159-9b9a-acf58d5bfe79 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:53.064420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a038767-a3ca-4159-9b9a-acf58d5bfe79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:53.084018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-5a038767-a3ca-4159-9b9a-acf58d5bfe79 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:53.085143 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5a038767-a3ca-4159-9b9a-acf58d5bfe79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:53.143142 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1395/2786] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:53.152916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5fda7-716b-4cf2-bba8-848fc64e6c8b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:53.154321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f7a5fda7-716b-4cf2-bba8-848fc64e6c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:53.188958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:53.214321 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5fda7-716b-4cf2-bba8-848fc64e6c8b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:53.217931 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bf267-3091-4c4f-a6bb-d12a2503f1f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:53.218536 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bf267-3091-4c4f-a6bb-d12a2503f1f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:53.218857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bf267-3091-4c4f-a6bb-d12a2503f1f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:53.219132 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bf267-3091-4c4f-a6bb-d12a2503f1f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:53.228485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77bf267-3091-4c4f-a6bb-d12a2503f1f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:53.229898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77bf267-3091-4c4f-a6bb-d12a2503f1f9 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:53.239699 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1392/2787] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:53.506269 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a038767-a3ca-4159-9b9a-acf58d5bfe79 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:53.507758 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-5a038767-a3ca-4159-9b9a-acf58d5bfe79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:53.521260 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a038767-a3ca-4159-9b9a-acf58d5bfe79 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:53.526019 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:53.526692 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:53.526987 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:53.527268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0 admin admin] PATH_INFO: `/v3/projects/d767fdb835ce4bcf937bb0e17aa5bf68` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:53.528523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d767fdb835ce4bcf937bb0e17aa5bf68)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:53.530573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:53.578038 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd767fdb835ce4bcf937bb0e17aa5bf68', 'request_id': 'req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:53.584822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd767fdb835ce4bcf937bb0e17aa5bf68', 'request_id': 'req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:53.590312 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd767fdb835ce4bcf937bb0e17aa5bf68', 'request_id': 'req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:53.603236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0 admin admin] The token cache is being invalidate because project d767fdb835ce4bcf937bb0e17aa5bf68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:28:53.603529 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:53.604152 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-d39fd6d0-27d0-44a6-a9ce-0262ca8828d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:28:53.605714 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1396/2788] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:28:53 2022] DELETE /identity/v3/projects/d767fdb835ce4bcf937bb0e17aa5bf68 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:53.610956 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b65444-2fab-41c2-ae03-982c9024740a None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:53.611247 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b65444-2fab-41c2-ae03-982c9024740a None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:53.611526 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b65444-2fab-41c2-ae03-982c9024740a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:53.631902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-f5b65444-2fab-41c2-ae03-982c9024740a None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:53.633161 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-f5b65444-2fab-41c2-ae03-982c9024740a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:53.692458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1393/2789] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:53.700649 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cab75-0a4d-4ff0-a5ff-607258feb1d7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:53.700945 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cab75-0a4d-4ff0-a5ff-607258feb1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:53.701227 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cab75-0a4d-4ff0-a5ff-607258feb1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:53.721399 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-791cab75-0a4d-4ff0-a5ff-607258feb1d7 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:53.724590 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-791cab75-0a4d-4ff0-a5ff-607258feb1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:53.734180 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1397/2790] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:53.738933 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b65444-2fab-41c2-ae03-982c9024740a None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:53.748604 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b65444-2fab-41c2-ae03-982c9024740a None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:53.753205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580dbfee-cd9c-4644-bf5d-f281fae9a625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:53.754488 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dbfee-cd9c-4644-bf5d-f281fae9a625 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:53.754810 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dbfee-cd9c-4644-bf5d-f281fae9a625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:53.755149 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580dbfee-cd9c-4644-bf5d-f281fae9a625 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:53.755851 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580dbfee-cd9c-4644-bf5d-f281fae9a625 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:28:53.756254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580dbfee-cd9c-4644-bf5d-f281fae9a625 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:53.757857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580dbfee-cd9c-4644-bf5d-f281fae9a625 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:53.769544 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1394/2791] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:28:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:28:53.773915 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cab75-0a4d-4ff0-a5ff-607258feb1d7 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:53.783412 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cab75-0a4d-4ff0-a5ff-607258feb1d7 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:53.788171 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4fe93-0af4-417c-b808-739e326b438d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:53.788795 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4fe93-0af4-417c-b808-739e326b438d admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:53.789079 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4fe93-0af4-417c-b808-739e326b438d admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:53.789353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4fe93-0af4-417c-b808-739e326b438d admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:53.790064 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4fe93-0af4-417c-b808-739e326b438d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:53.792034 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4fe93-0af4-417c-b808-739e326b438d admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:53.820463 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1398/2792] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:28:53 2022] POST /identity/v3/projects => generated 374 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:53.824626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580dbfee-cd9c-4644-bf5d-f281fae9a625 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:53.835081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580dbfee-cd9c-4644-bf5d-f281fae9a625 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:53.889839 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4546a-f47d-4c27-ac00-7f04b410f14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:53.890447 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4546a-f47d-4c27-ac00-7f04b410f14e admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:53.890749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4546a-f47d-4c27-ac00-7f04b410f14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:53.891025 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4546a-f47d-4c27-ac00-7f04b410f14e admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:53.891687 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b4546a-f47d-4c27-ac00-7f04b410f14e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:53.893583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b4546a-f47d-4c27-ac00-7f04b410f14e admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:53.914006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1395/2793] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:28:53 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:28:53.918122 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4fe93-0af4-417c-b808-739e326b438d admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:53.929161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4fe93-0af4-417c-b808-739e326b438d admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:53.934087 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d212f-8449-4050-a4ff-a295cce0b9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:53.934732 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d212f-8449-4050-a4ff-a295cce0b9ae admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:53.935022 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d212f-8449-4050-a4ff-a295cce0b9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:53.935301 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d212f-8449-4050-a4ff-a295cce0b9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:53.935999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d212f-8449-4050-a4ff-a295cce0b9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:53.937625 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d212f-8449-4050-a4ff-a295cce0b9ae admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:53.938975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:53.950234 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1399/2794] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:53 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:53.954471 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4546a-f47d-4c27-ac00-7f04b410f14e admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:53.964493 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4546a-f47d-4c27-ac00-7f04b410f14e admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:53.968992 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45f0ed-2a80-4dd0-ae44-f268b0f3db69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:53.969585 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45f0ed-2a80-4dd0-ae44-f268b0f3db69 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:53.969865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45f0ed-2a80-4dd0-ae44-f268b0f3db69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:53.970134 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45f0ed-2a80-4dd0-ae44-f268b0f3db69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:53.970846 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45f0ed-2a80-4dd0-ae44-f268b0f3db69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:53.973027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba45f0ed-2a80-4dd0-ae44-f268b0f3db69 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:53.983338 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1396/2795] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:53 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:53.987198 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d212f-8449-4050-a4ff-a295cce0b9ae admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:53.997735 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d212f-8449-4050-a4ff-a295cce0b9ae admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:54.002342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2f171-cd5b-4727-9d8a-c4df237220bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:54.003030 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2f171-cd5b-4727-9d8a-c4df237220bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:54.003318 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2f171-cd5b-4727-9d8a-c4df237220bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:54.004099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2f171-cd5b-4727-9d8a-c4df237220bf admin admin] PATH_INFO: `/v3/projects/7cf605507bae49a0a797b96db9da31bd/users/250e40933543431981bcb19db78fa144/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:54.018317 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2f171-cd5b-4727-9d8a-c4df237220bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cf605507bae49a0a797b96db9da31bd, user_id=250e40933543431981bcb19db78fa144, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:54.021864 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2f171-cd5b-4727-9d8a-c4df237220bf admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:54.034594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1400/2796] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:53 2022] PUT /identity/v3/projects/7cf605507bae49a0a797b96db9da31bd/users/250e40933543431981bcb19db78fa144/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:54.039525 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6d42-b1e0-43b6-8568-0cd5849b723c None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:54.039810 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6d42-b1e0-43b6-8568-0cd5849b723c None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:54.040083 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f6d42-b1e0-43b6-8568-0cd5849b723c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:54.058558 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-132f6d42-b1e0-43b6-8568-0cd5849b723c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:54.059785 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-132f6d42-b1e0-43b6-8568-0cd5849b723c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:54.093086 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:28:54.119758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1397/2797] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:54.129410 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2f171-cd5b-4727-9d8a-c4df237220bf admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:54.131323 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-8ee2f171-cd5b-4727-9d8a-c4df237220bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:54.187506 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2f171-cd5b-4727-9d8a-c4df237220bf admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:54.191647 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563e164-4be8-47da-ab2e-c20a9fdb93ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:54.192259 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563e164-4be8-47da-ab2e-c20a9fdb93ec service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:54.192540 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563e164-4be8-47da-ab2e-c20a9fdb93ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:54.192812 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563e164-4be8-47da-ab2e-c20a9fdb93ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:54.202679 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563e164-4be8-47da-ab2e-c20a9fdb93ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:54.204134 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563e164-4be8-47da-ab2e-c20a9fdb93ec service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:54.214258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1401/2798] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:28:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:28:58.121629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e0a1b-d806-4908-a26e-095cdfb15091 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:58.122090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e0a1b-d806-4908-a26e-095cdfb15091 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:58.122090 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e0a1b-d806-4908-a26e-095cdfb15091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:58.151252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-462e0a1b-d806-4908-a26e-095cdfb15091 None None] MFA Rules not processed for user `250e40933543431981bcb19db78fa144`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:28:58.152450 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-462e0a1b-d806-4908-a26e-095cdfb15091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:28:58.234109 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1398/2799] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:28:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:28:58.235531 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563e164-4be8-47da-ab2e-c20a9fdb93ec service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:58.246018 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563e164-4be8-47da-ab2e-c20a9fdb93ec service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:58.250909 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dad2f-88b0-4964-bc2f-5edc1d58e58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:58.251542 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dad2f-88b0-4964-bc2f-5edc1d58e58c admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:58.251828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dad2f-88b0-4964-bc2f-5edc1d58e58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:58.252105 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dad2f-88b0-4964-bc2f-5edc1d58e58c admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:58.252780 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03dad2f-88b0-4964-bc2f-5edc1d58e58c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:58.255379 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03dad2f-88b0-4964-bc2f-5edc1d58e58c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:58.282607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1402/2800] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:28:58 2022] POST /identity/v3/projects => generated 372 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:58.287621 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e0a1b-d806-4908-a26e-095cdfb15091 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:58.300162 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e0a1b-d806-4908-a26e-095cdfb15091 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:58.379424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db829d4e-c6dd-44ce-9e6a-60c0e6906c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:58.380058 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db829d4e-c6dd-44ce-9e6a-60c0e6906c6f admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:58.380346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db829d4e-c6dd-44ce-9e6a-60c0e6906c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:58.380627 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db829d4e-c6dd-44ce-9e6a-60c0e6906c6f admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:58.383526 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db829d4e-c6dd-44ce-9e6a-60c0e6906c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:58.385548 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db829d4e-c6dd-44ce-9e6a-60c0e6906c6f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:58.414096 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1399/2801] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:28:58 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:28:58.417791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dad2f-88b0-4964-bc2f-5edc1d58e58c admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:58.430998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dad2f-88b0-4964-bc2f-5edc1d58e58c admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:58.436087 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4ae22-df7e-457c-96d7-c49393ca5970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:58.436721 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4ae22-df7e-457c-96d7-c49393ca5970 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:58.437006 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4ae22-df7e-457c-96d7-c49393ca5970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:58.437291 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4ae22-df7e-457c-96d7-c49393ca5970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:58.437999 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4ae22-df7e-457c-96d7-c49393ca5970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:58.440316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4ae22-df7e-457c-96d7-c49393ca5970 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:58.452086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1403/2802] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:58 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:58.455994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db829d4e-c6dd-44ce-9e6a-60c0e6906c6f admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:58.466972 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db829d4e-c6dd-44ce-9e6a-60c0e6906c6f admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:58.472047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1289e3-dc2a-4b86-9b8b-789f222352ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:58.473282 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1289e3-dc2a-4b86-9b8b-789f222352ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:58.473777 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1289e3-dc2a-4b86-9b8b-789f222352ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:58.474068 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1289e3-dc2a-4b86-9b8b-789f222352ad admin admin] PATH_INFO: `/v3/projects/f69c93b17f034b04ad85b9c624890db8/users/eb7f209a3cdc4500bc1381ba4ae12946/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:58.494813 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1289e3-dc2a-4b86-9b8b-789f222352ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=f69c93b17f034b04ad85b9c624890db8, user_id=eb7f209a3cdc4500bc1381ba4ae12946, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:58.497994 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1289e3-dc2a-4b86-9b8b-789f222352ad admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:58.515410 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1400/2803] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:58 2022] PUT /identity/v3/projects/f69c93b17f034b04ad85b9c624890db8/users/eb7f209a3cdc4500bc1381ba4ae12946/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:28:58.519365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4ae22-df7e-457c-96d7-c49393ca5970 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:58.534633 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4ae22-df7e-457c-96d7-c49393ca5970 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:58.598594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c6000-7ae8-4bae-abd0-f32c6c7d0a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:58.599288 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c6000-7ae8-4bae-abd0-f32c6c7d0a7b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:58.599567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c6000-7ae8-4bae-abd0-f32c6c7d0a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:58.599841 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c6000-7ae8-4bae-abd0-f32c6c7d0a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:58.600551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c6000-7ae8-4bae-abd0-f32c6c7d0a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:58.602163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c6000-7ae8-4bae-abd0-f32c6c7d0a7b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:58.616378 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1404/2804] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:28:58 2022] GET /identity/v3/roles => generated 1427 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:28:58.620470 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1289e3-dc2a-4b86-9b8b-789f222352ad admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:28:58.631166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1289e3-dc2a-4b86-9b8b-789f222352ad admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:28:58.635896 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6267ad8-0fd7-4c77-a9d8-26699923b971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:28:58.636653 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6267ad8-0fd7-4c77-a9d8-26699923b971 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:28:58.636945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6267ad8-0fd7-4c77-a9d8-26699923b971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:28:58.637254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6267ad8-0fd7-4c77-a9d8-26699923b971 admin admin] PATH_INFO: `/v3/projects/f69c93b17f034b04ad85b9c624890db8/users/eb7f209a3cdc4500bc1381ba4ae12946/roles/f18a6c6595d849f7a012306d67d167e5` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:28:58.639288 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6267ad8-0fd7-4c77-a9d8-26699923b971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f69c93b17f034b04ad85b9c624890db8, user_id=eb7f209a3cdc4500bc1381ba4ae12946, role_id=f18a6c6595d849f7a012306d67d167e5)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:28:58.641950 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6267ad8-0fd7-4c77-a9d8-26699923b971 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:28:58.653503 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1401/2805] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:28:58 2022] PUT /identity/v3/projects/f69c93b17f034b04ad85b9c624890db8/users/eb7f209a3cdc4500bc1381ba4ae12946/roles/f18a6c6595d849f7a012306d67d167e5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:29:02.935500 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ee30d-f5b5-4e5c-a651-d197f5e0eca7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:02.936020 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ee30d-f5b5-4e5c-a651-d197f5e0eca7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:02.936120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ee30d-f5b5-4e5c-a651-d197f5e0eca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:02.974413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-ce9ee30d-f5b5-4e5c-a651-d197f5e0eca7 None None] MFA Rules not processed for user `eb7f209a3cdc4500bc1381ba4ae12946`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:02.975844 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-ce9ee30d-f5b5-4e5c-a651-d197f5e0eca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:03.006003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:03.055890 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1405/2806] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:29:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:03.079229 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6267ad8-0fd7-4c77-a9d8-26699923b971 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:03.083374 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-d6267ad8-0fd7-4c77-a9d8-26699923b971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:03.132087 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:03.199453 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6267ad8-0fd7-4c77-a9d8-26699923b971 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:03.207529 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc305f-5e6e-4f1c-9316-ab193aa21676 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:03.208601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc305f-5e6e-4f1c-9316-ab193aa21676 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:03.208601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc305f-5e6e-4f1c-9316-ab193aa21676 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:03.209047 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc305f-5e6e-4f1c-9316-ab193aa21676 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:03.284230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc305f-5e6e-4f1c-9316-ab193aa21676 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:03.286204 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc305f-5e6e-4f1c-9316-ab193aa21676 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:03.297441 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1402/2807] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:29:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:04.270319 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bd848-ca65-4dcf-8569-ee0bb6df5a67 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:04.270880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bd848-ca65-4dcf-8569-ee0bb6df5a67 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:04.270981 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bd848-ca65-4dcf-8569-ee0bb6df5a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:04.290980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-9b7bd848-ca65-4dcf-8569-ee0bb6df5a67 None None] MFA Rules not processed for user `250e40933543431981bcb19db78fa144`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:29:04.292207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9b7bd848-ca65-4dcf-8569-ee0bb6df5a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:04.299141 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1406/2808] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:29:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:29:13.292365 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=46 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:14.284232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=20 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:29:45.537678 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc305f-5e6e-4f1c-9316-ab193aa21676 service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:45.539202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-62fc305f-5e6e-4f1c-9316-ab193aa21676 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:45.597092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc305f-5e6e-4f1c-9316-ab193aa21676 service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:45.600672 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d94a6c-9a61-4dd4-a9ee-9902e4c78908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:45.601280 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d94a6c-9a61-4dd4-a9ee-9902e4c78908 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.601618 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d94a6c-9a61-4dd4-a9ee-9902e4c78908 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.602523 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d94a6c-9a61-4dd4-a9ee-9902e4c78908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.604336 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-84d94a6c-9a61-4dd4-a9ee-9902e4c78908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:45.618235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d94a6c-9a61-4dd4-a9ee-9902e4c78908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:45.619689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d94a6c-9a61-4dd4-a9ee-9902e4c78908 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:45.630555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1403/2809] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:29:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:45.759407 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bd848-ca65-4dcf-8569-ee0bb6df5a67 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:45.769045 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bd848-ca65-4dcf-8569-ee0bb6df5a67 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:45.773246 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8415d-ff52-48a3-99b5-8f2c458edabb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:45.774371 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8415d-ff52-48a3-99b5-8f2c458edabb service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:45.774654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8415d-ff52-48a3-99b5-8f2c458edabb service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:45.774996 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8415d-ff52-48a3-99b5-8f2c458edabb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:45.785249 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8415d-ff52-48a3-99b5-8f2c458edabb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:45.786685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8415d-ff52-48a3-99b5-8f2c458edabb service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:45.796742 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1407/2810] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:29:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:51.255166 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d94a6c-9a61-4dd4-a9ee-9902e4c78908 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:29:51.256577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-84d94a6c-9a61-4dd4-a9ee-9902e4c78908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:51.312258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d94a6c-9a61-4dd4-a9ee-9902e4c78908 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:29:51.315860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5ef14-47d9-42e0-9608-c0bff36efbfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:29:51.316475 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5ef14-47d9-42e0-9608-c0bff36efbfe service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:29:51.316753 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5ef14-47d9-42e0-9608-c0bff36efbfe service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:29:51.317025 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5ef14-47d9-42e0-9608-c0bff36efbfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:29:51.318816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c0a5ef14-47d9-42e0-9608-c0bff36efbfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:29:51.332682 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5ef14-47d9-42e0-9608-c0bff36efbfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:29:51.334107 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5ef14-47d9-42e0-9608-c0bff36efbfe service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:29:51.345694 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1404/2811] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:29:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:29:55.792478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:01.340886 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=52 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:30:07.531916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ac352-1427-4b9a-8919-0ac351d8012c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:30:07.532375 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ac352-1427-4b9a-8919-0ac351d8012c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:30:07.532475 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ac352-1427-4b9a-8919-0ac351d8012c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:30:07.551571 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-200ac352-1427-4b9a-8919-0ac351d8012c None None] MFA Rules not processed for user `49f98078ac424b1284932839be652265`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:30:07.552791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-200ac352-1427-4b9a-8919-0ac351d8012c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:30:07.606965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1408/2812] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:30:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:30:17.600116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=25 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:40.968957 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5ef14-47d9-42e0-9608-c0bff36efbfe service cinder] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:41.006763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5ef14-47d9-42e0-9608-c0bff36efbfe service cinder] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:41.010305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefd2a9-2c42-4a71-a686-58643b504885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:41.010944 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefd2a9-2c42-4a71-a686-58643b504885 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:41.011228 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefd2a9-2c42-4a71-a686-58643b504885 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:41.011506 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefd2a9-2c42-4a71-a686-58643b504885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:41.013300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-bbefd2a9-2c42-4a71-a686-58643b504885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:41.043802 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefd2a9-2c42-4a71-a686-58643b504885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:41.045737 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefd2a9-2c42-4a71-a686-58643b504885 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:41.053698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1405/2813] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:31:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:50.292044 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ac352-1427-4b9a-8919-0ac351d8012c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:50.342206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ac352-1427-4b9a-8919-0ac351d8012c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:50.349258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c92fb5-5d41-4ad6-82a3-afb3fe888a8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:50.349916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c92fb5-5d41-4ad6-82a3-afb3fe888a8c service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:50.350200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c92fb5-5d41-4ad6-82a3-afb3fe888a8c service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:50.350480 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c92fb5-5d41-4ad6-82a3-afb3fe888a8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:50.355067 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f6c92fb5-5d41-4ad6-82a3-afb3fe888a8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:50.369710 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c92fb5-5d41-4ad6-82a3-afb3fe888a8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:50.371081 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c92fb5-5d41-4ad6-82a3-afb3fe888a8c service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:50.395161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1409/2814] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:31:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:31:51.050230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=27 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:31:56.330587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefd2a9-2c42-4a71-a686-58643b504885 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:56.340355 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefd2a9-2c42-4a71-a686-58643b504885 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:56.344720 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010b6e5-7c45-49c0-bbbe-358db91d7075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:56.345347 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010b6e5-7c45-49c0-bbbe-358db91d7075 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:56.345629 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010b6e5-7c45-49c0-bbbe-358db91d7075 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:56.345906 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010b6e5-7c45-49c0-bbbe-358db91d7075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:56.347716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-4010b6e5-7c45-49c0-bbbe-358db91d7075 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:56.412081 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010b6e5-7c45-49c0-bbbe-358db91d7075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:56.413698 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010b6e5-7c45-49c0-bbbe-358db91d7075 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:56.424254 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1406/2815] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:31:59.566019 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c92fb5-5d41-4ad6-82a3-afb3fe888a8c service nova] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:59.567583 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f6c92fb5-5d41-4ad6-82a3-afb3fe888a8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:31:59.583450 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c92fb5-5d41-4ad6-82a3-afb3fe888a8c service nova] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:59.588000 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c629a4c-f6fe-4c78-b350-caadedafc055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:59.588632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c629a4c-f6fe-4c78-b350-caadedafc055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:59.588914 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c629a4c-f6fe-4c78-b350-caadedafc055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:59.589187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c629a4c-f6fe-4c78-b350-caadedafc055 admin admin] PATH_INFO: `/v3/users/49f98078ac424b1284932839be652265` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:59.590372 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c629a4c-f6fe-4c78-b350-caadedafc055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49f98078ac424b1284932839be652265)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:59.592326 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c629a4c-f6fe-4c78-b350-caadedafc055 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:59.639903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c629a4c-f6fe-4c78-b350-caadedafc055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49f98078ac424b1284932839be652265'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:59.640596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c629a4c-f6fe-4c78-b350-caadedafc055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49f98078ac424b1284932839be652265'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:59.647716 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c629a4c-f6fe-4c78-b350-caadedafc055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49f98078ac424b1284932839be652265'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:59.648389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c629a4c-f6fe-4c78-b350-caadedafc055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49f98078ac424b1284932839be652265'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:59.660120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0c629a4c-f6fe-4c78-b350-caadedafc055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49f98078ac424b1284932839be652265'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:59.672658 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1410/2816] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:31:59 2022] DELETE /identity/v3/users/49f98078ac424b1284932839be652265 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:31:59.850233 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010b6e5-7c45-49c0-bbbe-358db91d7075 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:31:59.861187 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010b6e5-7c45-49c0-bbbe-358db91d7075 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:31:59.918794 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62568f3-4430-4885-a09a-4e5dd85e0ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:31:59.919434 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62568f3-4430-4885-a09a-4e5dd85e0ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:31:59.919716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62568f3-4430-4885-a09a-4e5dd85e0ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:31:59.919996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62568f3-4430-4885-a09a-4e5dd85e0ada admin admin] PATH_INFO: `/v3/projects/89ef702cd1114e3eac72ff64e2d80b1c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:31:59.921806 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62568f3-4430-4885-a09a-4e5dd85e0ada admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ef702cd1114e3eac72ff64e2d80b1c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:31:59.923907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62568f3-4430-4885-a09a-4e5dd85e0ada admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:31:59.972405 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f62568f3-4430-4885-a09a-4e5dd85e0ada admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ef702cd1114e3eac72ff64e2d80b1c', 'request_id': 'req-f62568f3-4430-4885-a09a-4e5dd85e0ada'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:59.978458 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f62568f3-4430-4885-a09a-4e5dd85e0ada admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ef702cd1114e3eac72ff64e2d80b1c', 'request_id': 'req-f62568f3-4430-4885-a09a-4e5dd85e0ada'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:59.985774 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f62568f3-4430-4885-a09a-4e5dd85e0ada admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ef702cd1114e3eac72ff64e2d80b1c', 'request_id': 'req-f62568f3-4430-4885-a09a-4e5dd85e0ada'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:59.996259 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f62568f3-4430-4885-a09a-4e5dd85e0ada admin admin] The token cache is being invalidate because project 89ef702cd1114e3eac72ff64e2d80b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:31:59.996551 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f62568f3-4430-4885-a09a-4e5dd85e0ada admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:59.997173 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-f62568f3-4430-4885-a09a-4e5dd85e0ada admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:31:59.998820 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1407/2817] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:31:59 2022] DELETE /identity/v3/projects/89ef702cd1114e3eac72ff64e2d80b1c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:00.003685 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc4b68-aa3a-4ef7-aa82-fc2b468a8283 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:00.003978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc4b68-aa3a-4ef7-aa82-fc2b468a8283 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:00.004254 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc4b68-aa3a-4ef7-aa82-fc2b468a8283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:00.035906 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-13cc4b68-aa3a-4ef7-aa82-fc2b468a8283 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:00.039356 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-13cc4b68-aa3a-4ef7-aa82-fc2b468a8283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:00.098623 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1411/2818] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:00.107278 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275fc7d-bba4-47c5-8de5-e66617d3d919 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:00.107563 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275fc7d-bba4-47c5-8de5-e66617d3d919 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:00.107835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275fc7d-bba4-47c5-8de5-e66617d3d919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:00.128812 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-7275fc7d-bba4-47c5-8de5-e66617d3d919 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:00.130012 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-7275fc7d-bba4-47c5-8de5-e66617d3d919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:00.137902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1408/2819] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:00.142973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc4b68-aa3a-4ef7-aa82-fc2b468a8283 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:00.152642 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc4b68-aa3a-4ef7-aa82-fc2b468a8283 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:00.157151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cd53b-6787-4dba-84cf-a19e8c342237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:00.157753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cd53b-6787-4dba-84cf-a19e8c342237 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:00.158029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cd53b-6787-4dba-84cf-a19e8c342237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:00.158299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725cd53b-6787-4dba-84cf-a19e8c342237 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:00.159024 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725cd53b-6787-4dba-84cf-a19e8c342237 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:00.159327 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725cd53b-6787-4dba-84cf-a19e8c342237 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:00.160905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725cd53b-6787-4dba-84cf-a19e8c342237 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:00.172933 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1412/2820] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:32:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:32:00.177230 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275fc7d-bba4-47c5-8de5-e66617d3d919 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:00.186892 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275fc7d-bba4-47c5-8de5-e66617d3d919 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:00.191509 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543218c-8e6d-44d3-9c91-95baef27b906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:00.192100 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543218c-8e6d-44d3-9c91-95baef27b906 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:00.192378 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543218c-8e6d-44d3-9c91-95baef27b906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:00.192649 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543218c-8e6d-44d3-9c91-95baef27b906 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:00.193344 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543218c-8e6d-44d3-9c91-95baef27b906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:00.195276 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543218c-8e6d-44d3-9c91-95baef27b906 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:00.212236 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:00.225371 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1409/2821] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:32:00 2022] POST /identity/v3/projects => generated 386 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:00.229611 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cd53b-6787-4dba-84cf-a19e8c342237 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:00.241089 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725cd53b-6787-4dba-84cf-a19e8c342237 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:00.250973 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:00.299314 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ee169-952b-4b51-a486-401201b802b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:00.300041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ee169-952b-4b51-a486-401201b802b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:00.300333 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ee169-952b-4b51-a486-401201b802b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:00.300641 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ee169-952b-4b51-a486-401201b802b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:00.301487 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ee169-952b-4b51-a486-401201b802b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:00.303553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ee169-952b-4b51-a486-401201b802b3 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:00.325761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1413/2822] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:32:00 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:00.330477 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543218c-8e6d-44d3-9c91-95baef27b906 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:00.340577 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543218c-8e6d-44d3-9c91-95baef27b906 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:00.345148 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed84cba-14f9-492a-b9f3-386bf40b64ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:00.345747 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed84cba-14f9-492a-b9f3-386bf40b64ec admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:00.346024 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed84cba-14f9-492a-b9f3-386bf40b64ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:00.346295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed84cba-14f9-492a-b9f3-386bf40b64ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:00.347005 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed84cba-14f9-492a-b9f3-386bf40b64ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:00.348583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed84cba-14f9-492a-b9f3-386bf40b64ec admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:00.358954 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1410/2823] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:32:00 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:32:00.362852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ee169-952b-4b51-a486-401201b802b3 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:00.372676 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ee169-952b-4b51-a486-401201b802b3 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:00.379342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc98b0-82b8-4ea2-9f5a-000d49ea88e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:00.379970 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc98b0-82b8-4ea2-9f5a-000d49ea88e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:00.380359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc98b0-82b8-4ea2-9f5a-000d49ea88e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:00.380638 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc98b0-82b8-4ea2-9f5a-000d49ea88e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:00.381851 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc98b0-82b8-4ea2-9f5a-000d49ea88e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:00.383745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc98b0-82b8-4ea2-9f5a-000d49ea88e8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:00.395358 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1414/2824] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:32:00 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:00.399430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed84cba-14f9-492a-b9f3-386bf40b64ec admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:00.409609 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed84cba-14f9-492a-b9f3-386bf40b64ec admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:00.414643 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122584d-0650-4b4f-a74e-c699e6a76a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:00.415353 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122584d-0650-4b4f-a74e-c699e6a76a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:00.415638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122584d-0650-4b4f-a74e-c699e6a76a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:00.415918 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122584d-0650-4b4f-a74e-c699e6a76a70 admin admin] PATH_INFO: `/v3/projects/b3461561695b4e81aaa6628882c626fd/users/b7e1cc400a3a4e56901290597d6a7c55/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:00.430962 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122584d-0650-4b4f-a74e-c699e6a76a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3461561695b4e81aaa6628882c626fd, user_id=b7e1cc400a3a4e56901290597d6a7c55, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:00.433623 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122584d-0650-4b4f-a74e-c699e6a76a70 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:00.448222 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1411/2825] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:32:00 2022] PUT /identity/v3/projects/b3461561695b4e81aaa6628882c626fd/users/b7e1cc400a3a4e56901290597d6a7c55/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:32:00.459324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c279c71-abca-4be6-87e6-2d65c71c0439 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:00.459618 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c279c71-abca-4be6-87e6-2d65c71c0439 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:00.459902 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c279c71-abca-4be6-87e6-2d65c71c0439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:00.487901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-7c279c71-abca-4be6-87e6-2d65c71c0439 None None] MFA Rules not processed for user `b7e1cc400a3a4e56901290597d6a7c55`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:00.489116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-7c279c71-abca-4be6-87e6-2d65c71c0439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:00.556528 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1415/2826] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:00.568510 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122584d-0650-4b4f-a74e-c699e6a76a70 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:00.570218 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9122584d-0650-4b4f-a74e-c699e6a76a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:00.644057 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122584d-0650-4b4f-a74e-c699e6a76a70 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:00.649679 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d9daac-0411-4746-80f9-d40baf833670 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:00.650292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9daac-0411-4746-80f9-d40baf833670 service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:00.650576 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9daac-0411-4746-80f9-d40baf833670 service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:00.650885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9daac-0411-4746-80f9-d40baf833670 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:00.658779 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9daac-0411-4746-80f9-d40baf833670 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:00.660363 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d9daac-0411-4746-80f9-d40baf833670 service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:00.669163 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1412/2827] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:32:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:09.037103 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c279c71-abca-4be6-87e6-2d65c71c0439 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:09.102065 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c279c71-abca-4be6-87e6-2d65c71c0439 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:09.106687 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797f031-8d38-498d-a3fe-0b82b5f6ac50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:09.107336 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f031-8d38-498d-a3fe-0b82b5f6ac50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:09.107640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f031-8d38-498d-a3fe-0b82b5f6ac50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:09.107944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f031-8d38-498d-a3fe-0b82b5f6ac50 admin admin] PATH_INFO: `/v3/users/b7e1cc400a3a4e56901290597d6a7c55` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:09.110250 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797f031-8d38-498d-a3fe-0b82b5f6ac50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e1cc400a3a4e56901290597d6a7c55)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:09.114001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797f031-8d38-498d-a3fe-0b82b5f6ac50 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:09.154230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:09.186823 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9797f031-8d38-498d-a3fe-0b82b5f6ac50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e1cc400a3a4e56901290597d6a7c55'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.187486 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9797f031-8d38-498d-a3fe-0b82b5f6ac50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e1cc400a3a4e56901290597d6a7c55'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.196120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9797f031-8d38-498d-a3fe-0b82b5f6ac50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e1cc400a3a4e56901290597d6a7c55'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.196860 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9797f031-8d38-498d-a3fe-0b82b5f6ac50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e1cc400a3a4e56901290597d6a7c55'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.211005 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-9797f031-8d38-498d-a3fe-0b82b5f6ac50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e1cc400a3a4e56901290597d6a7c55'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.226003 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1416/2828] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:32:09 2022] DELETE /identity/v3/users/b7e1cc400a3a4e56901290597d6a7c55 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:09.230600 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce4768-b8ea-48d7-953d-aa0cdcae27a5 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:09.231147 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce4768-b8ea-48d7-953d-aa0cdcae27a5 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:09.231252 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce4768-b8ea-48d7-953d-aa0cdcae27a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:09.252153 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-a1ce4768-b8ea-48d7-953d-aa0cdcae27a5 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:09.253383 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a1ce4768-b8ea-48d7-953d-aa0cdcae27a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:09.314557 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1413/2829] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:09.324919 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797f031-8d38-498d-a3fe-0b82b5f6ac50 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:09.326328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9797f031-8d38-498d-a3fe-0b82b5f6ac50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:09.388657 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797f031-8d38-498d-a3fe-0b82b5f6ac50 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:09.392389 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a467431-64db-4954-acc1-cc4b7877fa17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:09.393016 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a467431-64db-4954-acc1-cc4b7877fa17 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:09.393299 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a467431-64db-4954-acc1-cc4b7877fa17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:09.393583 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a467431-64db-4954-acc1-cc4b7877fa17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:09.408791 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a467431-64db-4954-acc1-cc4b7877fa17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:09.411425 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a467431-64db-4954-acc1-cc4b7877fa17 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:09.423437 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1417/2830] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:09.733080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce4768-b8ea-48d7-953d-aa0cdcae27a5 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:09.735368 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-a1ce4768-b8ea-48d7-953d-aa0cdcae27a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:09.752576 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce4768-b8ea-48d7-953d-aa0cdcae27a5 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:09.757909 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc2785e-0e9b-4f6f-8064-9b72369470ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:09.758979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2785e-0e9b-4f6f-8064-9b72369470ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:09.758979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2785e-0e9b-4f6f-8064-9b72369470ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:09.759215 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc2785e-0e9b-4f6f-8064-9b72369470ac admin admin] PATH_INFO: `/v3/projects/b3461561695b4e81aaa6628882c626fd` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:09.760393 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2785e-0e9b-4f6f-8064-9b72369470ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3461561695b4e81aaa6628882c626fd)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:09.762342 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc2785e-0e9b-4f6f-8064-9b72369470ac admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:09.817190 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:09.826744 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8dc2785e-0e9b-4f6f-8064-9b72369470ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3461561695b4e81aaa6628882c626fd', 'request_id': 'req-8dc2785e-0e9b-4f6f-8064-9b72369470ac'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.837027 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8dc2785e-0e9b-4f6f-8064-9b72369470ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3461561695b4e81aaa6628882c626fd', 'request_id': 'req-8dc2785e-0e9b-4f6f-8064-9b72369470ac'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.845084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8dc2785e-0e9b-4f6f-8064-9b72369470ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3461561695b4e81aaa6628882c626fd', 'request_id': 'req-8dc2785e-0e9b-4f6f-8064-9b72369470ac'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.855996 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8dc2785e-0e9b-4f6f-8064-9b72369470ac admin admin] The token cache is being invalidate because project b3461561695b4e81aaa6628882c626fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:09.856291 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8dc2785e-0e9b-4f6f-8064-9b72369470ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.856948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-8dc2785e-0e9b-4f6f-8064-9b72369470ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:09.859084 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1414/2831] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:32:09 2022] DELETE /identity/v3/projects/b3461561695b4e81aaa6628882c626fd => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:09.864161 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf14268-b386-4ff9-8f27-c87ac519e5d9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:09.864449 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf14268-b386-4ff9-8f27-c87ac519e5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:09.864724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf14268-b386-4ff9-8f27-c87ac519e5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:09.885884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-dbf14268-b386-4ff9-8f27-c87ac519e5d9 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:09.887159 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-dbf14268-b386-4ff9-8f27-c87ac519e5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:09.953230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1418/2832] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:09.964340 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e08276-bc63-44c9-905d-688af0908570 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:09.964724 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e08276-bc63-44c9-905d-688af0908570 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:09.965008 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e08276-bc63-44c9-905d-688af0908570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:09.986575 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-e9e08276-bc63-44c9-905d-688af0908570 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:09.987925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e9e08276-bc63-44c9-905d-688af0908570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:09.996020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1415/2833] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:10.001086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf14268-b386-4ff9-8f27-c87ac519e5d9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.013180 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf14268-b386-4ff9-8f27-c87ac519e5d9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.017954 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230910f-9631-42ff-a16e-2c90bfcd7a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.018614 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230910f-9631-42ff-a16e-2c90bfcd7a48 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.018930 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230910f-9631-42ff-a16e-2c90bfcd7a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.019206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230910f-9631-42ff-a16e-2c90bfcd7a48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.019937 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230910f-9631-42ff-a16e-2c90bfcd7a48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:10.020247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230910f-9631-42ff-a16e-2c90bfcd7a48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.021898 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230910f-9631-42ff-a16e-2c90bfcd7a48 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.034944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1419/2834] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:32:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:32:10.039400 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e08276-bc63-44c9-905d-688af0908570 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.049763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e08276-bc63-44c9-905d-688af0908570 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.054683 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f2fcf-5898-49bf-829a-43e36debb39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.055310 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f2fcf-5898-49bf-829a-43e36debb39b admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.055596 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f2fcf-5898-49bf-829a-43e36debb39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.055914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f2fcf-5898-49bf-829a-43e36debb39b admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.056541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f2fcf-5898-49bf-829a-43e36debb39b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.058459 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9f2fcf-5898-49bf-829a-43e36debb39b admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.088456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1416/2835] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:32:10 2022] POST /identity/v3/projects => generated 386 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:10.093197 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230910f-9631-42ff-a16e-2c90bfcd7a48 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.103678 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230910f-9631-42ff-a16e-2c90bfcd7a48 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.139305 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:10.172179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2976bab-592c-4d4f-bb1a-8e57f911ab26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.172802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2976bab-592c-4d4f-bb1a-8e57f911ab26 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.173086 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2976bab-592c-4d4f-bb1a-8e57f911ab26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.173364 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2976bab-592c-4d4f-bb1a-8e57f911ab26 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.174041 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2976bab-592c-4d4f-bb1a-8e57f911ab26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.175991 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2976bab-592c-4d4f-bb1a-8e57f911ab26 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.200682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1420/2836] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:32:10 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:10.205456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f2fcf-5898-49bf-829a-43e36debb39b admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.215865 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f2fcf-5898-49bf-829a-43e36debb39b admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.221266 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e3852-d829-4234-82f4-6039b7383fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.221958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e3852-d829-4234-82f4-6039b7383fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.222205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e3852-d829-4234-82f4-6039b7383fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.222479 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e3852-d829-4234-82f4-6039b7383fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.223273 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e3852-d829-4234-82f4-6039b7383fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.224995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e3852-d829-4234-82f4-6039b7383fb2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.235892 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1417/2837] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:32:10 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:10.242126 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2976bab-592c-4d4f-bb1a-8e57f911ab26 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.255855 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2976bab-592c-4d4f-bb1a-8e57f911ab26 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.261073 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e47a4-c339-4481-9db9-404d8d558ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.261743 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e47a4-c339-4481-9db9-404d8d558ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.262085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e47a4-c339-4481-9db9-404d8d558ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.262365 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e47a4-c339-4481-9db9-404d8d558ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.263170 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e47a4-c339-4481-9db9-404d8d558ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.265382 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e47a4-c339-4481-9db9-404d8d558ee6 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.278742 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1421/2838] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:32:10 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:10.283518 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e3852-d829-4234-82f4-6039b7383fb2 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.294448 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e3852-d829-4234-82f4-6039b7383fb2 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.299301 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33a6e4-93ae-4514-a0c9-e5b1c0588548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.300006 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33a6e4-93ae-4514-a0c9-e5b1c0588548 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.300292 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33a6e4-93ae-4514-a0c9-e5b1c0588548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.300566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33a6e4-93ae-4514-a0c9-e5b1c0588548 admin admin] PATH_INFO: `/v3/projects/db7d2bc790f9455dab130c6d52ccdd9c/users/07490ac79fe64d3789c466cf139afa79/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.322099 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33a6e4-93ae-4514-a0c9-e5b1c0588548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db7d2bc790f9455dab130c6d52ccdd9c, user_id=07490ac79fe64d3789c466cf139afa79, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.325269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33a6e4-93ae-4514-a0c9-e5b1c0588548 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.345284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1418/2839] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:32:10 2022] PUT /identity/v3/projects/db7d2bc790f9455dab130c6d52ccdd9c/users/07490ac79fe64d3789c466cf139afa79/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:10.413459 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cb19c-667e-4321-a137-add5cc22ffc1 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.413750 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cb19c-667e-4321-a137-add5cc22ffc1 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.414029 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cb19c-667e-4321-a137-add5cc22ffc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.444082 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3c0cb19c-667e-4321-a137-add5cc22ffc1 None None] MFA Rules not processed for user `07490ac79fe64d3789c466cf139afa79`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:10.445339 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3c0cb19c-667e-4321-a137-add5cc22ffc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:10.511889 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1422/2840] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:10.523533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33a6e4-93ae-4514-a0c9-e5b1c0588548 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:10.525304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9f33a6e4-93ae-4514-a0c9-e5b1c0588548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:10.608080 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33a6e4-93ae-4514-a0c9-e5b1c0588548 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:10.611920 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2c0f4-b859-436b-b991-493f83af51f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:10.612541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2c0f4-b859-436b-b991-493f83af51f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:10.612821 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2c0f4-b859-436b-b991-493f83af51f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:10.613098 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2c0f4-b859-436b-b991-493f83af51f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:10.622055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff2c0f4-b859-436b-b991-493f83af51f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:10.623541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff2c0f4-b859-436b-b991-493f83af51f8 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:10.634530 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1419/2841] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:32:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:19.134446 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cb19c-667e-4321-a137-add5cc22ffc1 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:19.199163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cb19c-667e-4321-a137-add5cc22ffc1 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:19.204493 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72035a60-8988-43fe-b341-0c4aacabf0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:19.205138 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72035a60-8988-43fe-b341-0c4aacabf0e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:19.205419 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72035a60-8988-43fe-b341-0c4aacabf0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:19.205694 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72035a60-8988-43fe-b341-0c4aacabf0e8 admin admin] PATH_INFO: `/v3/users/07490ac79fe64d3789c466cf139afa79` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:19.206901 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72035a60-8988-43fe-b341-0c4aacabf0e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07490ac79fe64d3789c466cf139afa79)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:19.208840 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72035a60-8988-43fe-b341-0c4aacabf0e8 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:19.255129 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:19.270230 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-72035a60-8988-43fe-b341-0c4aacabf0e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07490ac79fe64d3789c466cf139afa79'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:19.271419 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-72035a60-8988-43fe-b341-0c4aacabf0e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07490ac79fe64d3789c466cf139afa79'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:19.279543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-72035a60-8988-43fe-b341-0c4aacabf0e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07490ac79fe64d3789c466cf139afa79'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:19.280748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-72035a60-8988-43fe-b341-0c4aacabf0e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07490ac79fe64d3789c466cf139afa79'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:19.292916 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-72035a60-8988-43fe-b341-0c4aacabf0e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07490ac79fe64d3789c466cf139afa79'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:19.307292 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1423/2842] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:32:19 2022] DELETE /identity/v3/users/07490ac79fe64d3789c466cf139afa79 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:19.311843 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c92bf-7311-4744-abfc-f5fbc6cb53dd None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:19.312505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c92bf-7311-4744-abfc-f5fbc6cb53dd None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:19.312505 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c92bf-7311-4744-abfc-f5fbc6cb53dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:19.333113 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-316c92bf-7311-4744-abfc-f5fbc6cb53dd None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:19.334346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-316c92bf-7311-4744-abfc-f5fbc6cb53dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:19.384205 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:19.410938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1420/2843] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:19.420816 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72035a60-8988-43fe-b341-0c4aacabf0e8 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:19.422275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-72035a60-8988-43fe-b341-0c4aacabf0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:19.483359 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72035a60-8988-43fe-b341-0c4aacabf0e8 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:19.487213 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b60429-06c1-4212-921d-fd0922df71a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:19.487859 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b60429-06c1-4212-921d-fd0922df71a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:19.488141 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b60429-06c1-4212-921d-fd0922df71a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:19.488426 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b60429-06c1-4212-921d-fd0922df71a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:19.498320 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b60429-06c1-4212-921d-fd0922df71a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:19.499822 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b60429-06c1-4212-921d-fd0922df71a6 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:19.509851 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1424/2844] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:19.817555 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c92bf-7311-4744-abfc-f5fbc6cb53dd None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:19.820326 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-316c92bf-7311-4744-abfc-f5fbc6cb53dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:19.850700 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c92bf-7311-4744-abfc-f5fbc6cb53dd None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:19.866150 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef087811-8bd5-4cd6-bb68-73a024d78940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:19.866991 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef087811-8bd5-4cd6-bb68-73a024d78940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:19.867323 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef087811-8bd5-4cd6-bb68-73a024d78940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:19.867705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef087811-8bd5-4cd6-bb68-73a024d78940 admin admin] PATH_INFO: `/v3/projects/db7d2bc790f9455dab130c6d52ccdd9c` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:19.869693 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef087811-8bd5-4cd6-bb68-73a024d78940 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db7d2bc790f9455dab130c6d52ccdd9c)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:19.872512 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef087811-8bd5-4cd6-bb68-73a024d78940 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:19.927415 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ef087811-8bd5-4cd6-bb68-73a024d78940 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db7d2bc790f9455dab130c6d52ccdd9c', 'request_id': 'req-ef087811-8bd5-4cd6-bb68-73a024d78940'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:19.934850 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ef087811-8bd5-4cd6-bb68-73a024d78940 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db7d2bc790f9455dab130c6d52ccdd9c', 'request_id': 'req-ef087811-8bd5-4cd6-bb68-73a024d78940'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:19.943591 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ef087811-8bd5-4cd6-bb68-73a024d78940 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db7d2bc790f9455dab130c6d52ccdd9c', 'request_id': 'req-ef087811-8bd5-4cd6-bb68-73a024d78940'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:19.956516 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ef087811-8bd5-4cd6-bb68-73a024d78940 admin admin] The token cache is being invalidate because project db7d2bc790f9455dab130c6d52ccdd9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:19.956806 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ef087811-8bd5-4cd6-bb68-73a024d78940 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:19.957467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ef087811-8bd5-4cd6-bb68-73a024d78940 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:19.959039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1421/2845] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:32:19 2022] DELETE /identity/v3/projects/db7d2bc790f9455dab130c6d52ccdd9c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:19.964247 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1b437-071f-42b7-8fb5-6403e2528590 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:19.964686 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1b437-071f-42b7-8fb5-6403e2528590 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:19.964969 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa1b437-071f-42b7-8fb5-6403e2528590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:19.989592 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-6fa1b437-071f-42b7-8fb5-6403e2528590 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:19.991677 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-6fa1b437-071f-42b7-8fb5-6403e2528590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:20.058017 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1425/2846] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:20.065667 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46797ff-6469-45ee-b784-dcd5907fd3f6 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:20.065959 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46797ff-6469-45ee-b784-dcd5907fd3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:20.066242 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46797ff-6469-45ee-b784-dcd5907fd3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:20.093656 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-b46797ff-6469-45ee-b784-dcd5907fd3f6 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:20.095565 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-b46797ff-6469-45ee-b784-dcd5907fd3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:20.111807 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1422/2847] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:20.116270 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1b437-071f-42b7-8fb5-6403e2528590 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:20.127708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa1b437-071f-42b7-8fb5-6403e2528590 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:20.133533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbced5fe-c216-4ef8-8180-772b432f6733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:20.134116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbced5fe-c216-4ef8-8180-772b432f6733 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:20.134470 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbced5fe-c216-4ef8-8180-772b432f6733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:20.135192 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbced5fe-c216-4ef8-8180-772b432f6733 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:20.135461 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbced5fe-c216-4ef8-8180-772b432f6733 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:20.135765 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbced5fe-c216-4ef8-8180-772b432f6733 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:20.137360 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbced5fe-c216-4ef8-8180-772b432f6733 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:20.152279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1426/2848] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:32:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:32:20.157300 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46797ff-6469-45ee-b784-dcd5907fd3f6 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:20.166910 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46797ff-6469-45ee-b784-dcd5907fd3f6 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:20.172868 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c183128-d7f6-4990-844c-06e84b63feb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:20.173487 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c183128-d7f6-4990-844c-06e84b63feb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:20.173766 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c183128-d7f6-4990-844c-06e84b63feb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:20.174039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c183128-d7f6-4990-844c-06e84b63feb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:20.174736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c183128-d7f6-4990-844c-06e84b63feb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:20.176638 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c183128-d7f6-4990-844c-06e84b63feb6 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:20.206351 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1423/2849] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:32:20 2022] POST /identity/v3/projects => generated 358 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:32:20.210761 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbced5fe-c216-4ef8-8180-772b432f6733 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:20.221708 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbced5fe-c216-4ef8-8180-772b432f6733 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:20.277286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:20.294150 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83c1ec-2c77-414b-b36c-d872dd6f3600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:20.294896 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83c1ec-2c77-414b-b36c-d872dd6f3600 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:20.295191 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83c1ec-2c77-414b-b36c-d872dd6f3600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:20.295505 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83c1ec-2c77-414b-b36c-d872dd6f3600 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:20.296201 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83c1ec-2c77-414b-b36c-d872dd6f3600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:20.298151 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83c1ec-2c77-414b-b36c-d872dd6f3600 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:20.326517 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1427/2850] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:32:20 2022] POST /identity/v3/users => generated 280 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 15:32:20.331364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c183128-d7f6-4990-844c-06e84b63feb6 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:20.343592 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c183128-d7f6-4990-844c-06e84b63feb6 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:20.349945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e11f59-f571-435a-a44a-24af73a24701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:20.350576 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e11f59-f571-435a-a44a-24af73a24701 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:20.350898 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e11f59-f571-435a-a44a-24af73a24701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:20.351175 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e11f59-f571-435a-a44a-24af73a24701 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:20.351888 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e11f59-f571-435a-a44a-24af73a24701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:20.353515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e11f59-f571-435a-a44a-24af73a24701 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:20.365603 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1424/2851] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:32:20 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:20.369968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83c1ec-2c77-414b-b36c-d872dd6f3600 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:20.388604 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83c1ec-2c77-414b-b36c-d872dd6f3600 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:20.394534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90b5a9-f45e-44b9-a4f4-83ec1d8e813b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:20.395206 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90b5a9-f45e-44b9-a4f4-83ec1d8e813b admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:20.395482 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90b5a9-f45e-44b9-a4f4-83ec1d8e813b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:20.395753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90b5a9-f45e-44b9-a4f4-83ec1d8e813b admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:20.396473 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90b5a9-f45e-44b9-a4f4-83ec1d8e813b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:20.398431 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90b5a9-f45e-44b9-a4f4-83ec1d8e813b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:20.410021 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1428/2852] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:32:20 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:20.414248 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e11f59-f571-435a-a44a-24af73a24701 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:20.424601 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e11f59-f571-435a-a44a-24af73a24701 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:20.429468 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c24024-5edd-44b5-8052-c3a0aeccbb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:20.430137 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c24024-5edd-44b5-8052-c3a0aeccbb87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:20.430420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c24024-5edd-44b5-8052-c3a0aeccbb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:20.430736 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c24024-5edd-44b5-8052-c3a0aeccbb87 admin admin] PATH_INFO: `/v3/projects/0d3b5ad94b8443648639edfce725ea7d/users/0b6dc93bfec144f9adce7fb8ad5e665f/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:20.446892 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c24024-5edd-44b5-8052-c3a0aeccbb87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d3b5ad94b8443648639edfce725ea7d, user_id=0b6dc93bfec144f9adce7fb8ad5e665f, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:20.449823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c24024-5edd-44b5-8052-c3a0aeccbb87 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:20.465472 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1425/2853] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:32:20 2022] PUT /identity/v3/projects/0d3b5ad94b8443648639edfce725ea7d/users/0b6dc93bfec144f9adce7fb8ad5e665f/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:20.476980 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06831841-9c20-4b0a-9d2a-4cab240c4297 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:20.477273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06831841-9c20-4b0a-9d2a-4cab240c4297 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:20.477573 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06831841-9c20-4b0a-9d2a-4cab240c4297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:20.506842 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-06831841-9c20-4b0a-9d2a-4cab240c4297 None None] MFA Rules not processed for user `0b6dc93bfec144f9adce7fb8ad5e665f`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:20.508085 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-06831841-9c20-4b0a-9d2a-4cab240c4297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:20.577975 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1429/2854] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:20.614092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c24024-5edd-44b5-8052-c3a0aeccbb87 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:20.616145 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-e6c24024-5edd-44b5-8052-c3a0aeccbb87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:20.681295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c24024-5edd-44b5-8052-c3a0aeccbb87 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:20.684993 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c0561-7efd-4ae8-8027-5b1e9422ba2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:20.685616 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c0561-7efd-4ae8-8027-5b1e9422ba2e service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:20.685901 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c0561-7efd-4ae8-8027-5b1e9422ba2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:20.687011 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c0561-7efd-4ae8-8027-5b1e9422ba2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:20.695995 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c0561-7efd-4ae8-8027-5b1e9422ba2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:20.697446 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c0561-7efd-4ae8-8027-5b1e9422ba2e service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:20.707086 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1426/2855] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:32:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:20.722094 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06831841-9c20-4b0a-9d2a-4cab240c4297 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:20.790587 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06831841-9c20-4b0a-9d2a-4cab240c4297 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:20.795215 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf73060-15e1-4f70-aa20-5d1e957cecf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:20.796387 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf73060-15e1-4f70-aa20-5d1e957cecf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:20.796667 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf73060-15e1-4f70-aa20-5d1e957cecf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:20.796946 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf73060-15e1-4f70-aa20-5d1e957cecf4 admin admin] PATH_INFO: `/v3/users/0b6dc93bfec144f9adce7fb8ad5e665f` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:20.798148 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf73060-15e1-4f70-aa20-5d1e957cecf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6dc93bfec144f9adce7fb8ad5e665f)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:20.800097 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf73060-15e1-4f70-aa20-5d1e957cecf4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:20.852671 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2bf73060-15e1-4f70-aa20-5d1e957cecf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6dc93bfec144f9adce7fb8ad5e665f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:20.853353 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2bf73060-15e1-4f70-aa20-5d1e957cecf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6dc93bfec144f9adce7fb8ad5e665f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:20.860872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2bf73060-15e1-4f70-aa20-5d1e957cecf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6dc93bfec144f9adce7fb8ad5e665f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:20.861550 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2bf73060-15e1-4f70-aa20-5d1e957cecf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6dc93bfec144f9adce7fb8ad5e665f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:20.863460 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:20.876594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2bf73060-15e1-4f70-aa20-5d1e957cecf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6dc93bfec144f9adce7fb8ad5e665f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:20.890096 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1430/2856] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:32:20 2022] DELETE /identity/v3/users/0b6dc93bfec144f9adce7fb8ad5e665f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:20.894690 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69131c54-b54e-4c96-bad2-24234149a7d1 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:20.895043 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69131c54-b54e-4c96-bad2-24234149a7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:20.895379 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69131c54-b54e-4c96-bad2-24234149a7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:20.915265 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-69131c54-b54e-4c96-bad2-24234149a7d1 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:20.916499 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-69131c54-b54e-4c96-bad2-24234149a7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:20.923945 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:20.991914 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1427/2857] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:21.002865 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf73060-15e1-4f70-aa20-5d1e957cecf4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:21.004344 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-2bf73060-15e1-4f70-aa20-5d1e957cecf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:21.071214 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf73060-15e1-4f70-aa20-5d1e957cecf4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:21.077286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f83eb-8b8c-409c-88a2-28e13850e112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:21.077926 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f83eb-8b8c-409c-88a2-28e13850e112 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:21.078202 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f83eb-8b8c-409c-88a2-28e13850e112 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:21.078471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f83eb-8b8c-409c-88a2-28e13850e112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:21.088885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f83eb-8b8c-409c-88a2-28e13850e112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:21.090326 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f83eb-8b8c-409c-88a2-28e13850e112 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:21.100802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1431/2858] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:21.394069 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69131c54-b54e-4c96-bad2-24234149a7d1 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:21.395840 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-69131c54-b54e-4c96-bad2-24234149a7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:21.411208 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69131c54-b54e-4c96-bad2-24234149a7d1 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:21.419718 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:21.421196 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:21.421487 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:21.421766 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89 admin admin] PATH_INFO: `/v3/projects/0d3b5ad94b8443648639edfce725ea7d` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:21.423073 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d3b5ad94b8443648639edfce725ea7d)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:21.425020 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:21.486625 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d3b5ad94b8443648639edfce725ea7d', 'request_id': 'req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:21.494303 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d3b5ad94b8443648639edfce725ea7d', 'request_id': 'req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:21.503513 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d3b5ad94b8443648639edfce725ea7d', 'request_id': 'req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:21.518788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89 admin admin] The token cache is being invalidate because project 0d3b5ad94b8443648639edfce725ea7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:21.519154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:21.519833 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-325c82ef-efa7-4cd9-8b9c-2b7550c0de89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:21.521558 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1428/2859] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:32:21 2022] DELETE /identity/v3/projects/0d3b5ad94b8443648639edfce725ea7d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:21.526354 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb446401-95df-489c-bedd-665717ee92b9 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:21.526944 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb446401-95df-489c-bedd-665717ee92b9 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:21.527062 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb446401-95df-489c-bedd-665717ee92b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:21.550748 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bb446401-95df-489c-bedd-665717ee92b9 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:21.552083 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bb446401-95df-489c-bedd-665717ee92b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:21.617757 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1432/2860] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:21.626426 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b2637-ffd7-462e-b40a-585a24386e70 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:21.626758 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b2637-ffd7-462e-b40a-585a24386e70 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:21.627040 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b2637-ffd7-462e-b40a-585a24386e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:21.648295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-9d5b2637-ffd7-462e-b40a-585a24386e70 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:21.649546 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-9d5b2637-ffd7-462e-b40a-585a24386e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:21.657902 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1429/2861] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:21.662419 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb446401-95df-489c-bedd-665717ee92b9 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:21.673772 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb446401-95df-489c-bedd-665717ee92b9 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:21.679890 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52685a-15dd-4657-997b-52bf2d024660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:21.680536 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52685a-15dd-4657-997b-52bf2d024660 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:21.680824 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52685a-15dd-4657-997b-52bf2d024660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:21.681102 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52685a-15dd-4657-997b-52bf2d024660 admin admin] PATH_INFO: `/v3/domains` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:21.681814 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52685a-15dd-4657-997b-52bf2d024660 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80358) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:21.682126 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52685a-15dd-4657-997b-52bf2d024660 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:21.683777 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52685a-15dd-4657-997b-52bf2d024660 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:21.696594 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1433/2862] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:32:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:32:21.701417 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b2637-ffd7-462e-b40a-585a24386e70 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:21.713224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b2637-ffd7-462e-b40a-585a24386e70 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:21.718051 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de182945-98fc-4d86-8170-2f9aa8625c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:21.718705 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de182945-98fc-4d86-8170-2f9aa8625c78 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:21.718987 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de182945-98fc-4d86-8170-2f9aa8625c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:21.719260 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de182945-98fc-4d86-8170-2f9aa8625c78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:21.719938 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de182945-98fc-4d86-8170-2f9aa8625c78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:21.721856 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de182945-98fc-4d86-8170-2f9aa8625c78 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:21.767430 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1430/2863] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:32:21 2022] POST /identity/v3/projects => generated 378 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:21.768565 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52685a-15dd-4657-997b-52bf2d024660 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:21.779417 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52685a-15dd-4657-997b-52bf2d024660 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:21.845237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc21de-3da8-4270-bc90-44465f3c3171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:21.845892 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc21de-3da8-4270-bc90-44465f3c3171 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:21.846179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc21de-3da8-4270-bc90-44465f3c3171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:21.846458 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc21de-3da8-4270-bc90-44465f3c3171 admin admin] PATH_INFO: `/v3/users` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:21.847185 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc21de-3da8-4270-bc90-44465f3c3171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:21.849207 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc21de-3da8-4270-bc90-44465f3c3171 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:21.862140 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:21.875084 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1434/2864] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:32:21 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:21.879795 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de182945-98fc-4d86-8170-2f9aa8625c78 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:21.894234 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de182945-98fc-4d86-8170-2f9aa8625c78 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:21.898961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab915cac-bf5b-4835-b2e1-f94238ec5ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:21.899581 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab915cac-bf5b-4835-b2e1-f94238ec5ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:21.899859 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab915cac-bf5b-4835-b2e1-f94238ec5ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:21.900136 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab915cac-bf5b-4835-b2e1-f94238ec5ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:21.900877 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab915cac-bf5b-4835-b2e1-f94238ec5ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:21.902485 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab915cac-bf5b-4835-b2e1-f94238ec5ee3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:21.915264 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1431/2865] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:32:21 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:21.919716 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc21de-3da8-4270-bc90-44465f3c3171 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:21.930675 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc21de-3da8-4270-bc90-44465f3c3171 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:21.935448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c027c-db29-4818-8d9f-8651f9818f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:21.936120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c027c-db29-4818-8d9f-8651f9818f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:21.936408 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c027c-db29-4818-8d9f-8651f9818f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:21.936690 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c027c-db29-4818-8d9f-8651f9818f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:21.937395 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c027c-db29-4818-8d9f-8651f9818f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:21.939047 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c027c-db29-4818-8d9f-8651f9818f54 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:21.950938 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1435/2866] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:32:21 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 15:32:21.955351 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab915cac-bf5b-4835-b2e1-f94238ec5ee3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:21.965771 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab915cac-bf5b-4835-b2e1-f94238ec5ee3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:21.971242 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dc991-a496-4071-8ced-67950e0ed3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:21.971917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dc991-a496-4071-8ced-67950e0ed3ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:21.972209 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dc991-a496-4071-8ced-67950e0ed3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:21.972483 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dc991-a496-4071-8ced-67950e0ed3ba admin admin] PATH_INFO: `/v3/projects/e3d9d9c6b7bb43e98a0a3865e44a9626/users/9040a1d570d54c9cbadfca9cb4f7710d/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:21.988976 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0dc991-a496-4071-8ced-67950e0ed3ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d9d9c6b7bb43e98a0a3865e44a9626, user_id=9040a1d570d54c9cbadfca9cb4f7710d, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:21.991714 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0dc991-a496-4071-8ced-67950e0ed3ba admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:22.007829 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1432/2867] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:32:21 2022] PUT /identity/v3/projects/e3d9d9c6b7bb43e98a0a3865e44a9626/users/9040a1d570d54c9cbadfca9cb4f7710d/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:22.018325 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21debfb-2173-4ea4-9380-e2611a4ce8e7 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:22.018560 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21debfb-2173-4ea4-9380-e2611a4ce8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:22.018931 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21debfb-2173-4ea4-9380-e2611a4ce8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:22.052191 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-f21debfb-2173-4ea4-9380-e2611a4ce8e7 None None] MFA Rules not processed for user `9040a1d570d54c9cbadfca9cb4f7710d`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:22.053640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f21debfb-2173-4ea4-9380-e2611a4ce8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:22.121955 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1436/2868] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:22.133622 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dc991-a496-4071-8ced-67950e0ed3ba admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:22.135373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-8e0dc991-a496-4071-8ced-67950e0ed3ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:22.176119 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:22.206092 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dc991-a496-4071-8ced-67950e0ed3ba admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:22.209925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87f360-c54b-41a9-bd86-3767106f5189 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:22.210559 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87f360-c54b-41a9-bd86-3767106f5189 service cinder] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:22.210875 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87f360-c54b-41a9-bd86-3767106f5189 service cinder] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:22.211154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87f360-c54b-41a9-bd86-3767106f5189 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:22.221467 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87f360-c54b-41a9-bd86-3767106f5189 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:22.222965 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87f360-c54b-41a9-bd86-3767106f5189 service cinder] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:22.233216 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1433/2869] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:32:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:24.839380 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3c36c-d4dd-468a-8d83-f010def9d7c0 None None] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:24.839877 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3c36c-d4dd-468a-8d83-f010def9d7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:24.839983 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3c36c-d4dd-468a-8d83-f010def9d7c0 None None] PATH_INFO: `/` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:24.841375 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1437/2870] 10.222.0.100 () {54 vars in 926 bytes} [Mon Aug 1 15:32:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:32:24.849337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686f296-dbdf-4d36-b911-89318dc564c9 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:24.849891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686f296-dbdf-4d36-b911-89318dc564c9 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:24.850178 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686f296-dbdf-4d36-b911-89318dc564c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:24.923719 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1686f296-dbdf-4d36-b911-89318dc564c9 None None] MFA Rules not processed for user `e7f757eabcb84de6ab042e0ac5a8158c`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:24.923719 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1686f296-dbdf-4d36-b911-89318dc564c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:24.974337 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1434/2871] 10.222.0.100 () {58 vars in 1023 bytes} [Mon Aug 1 15:32:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 15:32:24.986828 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3c36c-d4dd-468a-8d83-f010def9d7c0 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:24.989286 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-11f3c36c-d4dd-468a-8d83-f010def9d7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:25.003978 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3c36c-d4dd-468a-8d83-f010def9d7c0 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:25.007998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c39cc-648f-4711-9166-3ec10561bedb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:25.008625 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c39cc-648f-4711-9166-3ec10561bedb service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.011334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c39cc-648f-4711-9166-3ec10561bedb service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.011334 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c39cc-648f-4711-9166-3ec10561bedb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.017784 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c39cc-648f-4711-9166-3ec10561bedb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:25.019262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c39cc-648f-4711-9166-3ec10561bedb service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:25.028264 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1438/2872] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:32:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:32:25.117359 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24497cd3-3db1-457e-812f-38da49c6d049 None None] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.117636 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24497cd3-3db1-457e-812f-38da49c6d049 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.117916 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24497cd3-3db1-457e-812f-38da49c6d049 None None] PATH_INFO: `/` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.119370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1435/2873] 10.222.0.100 () {54 vars in 926 bytes} [Mon Aug 1 15:32:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 01 15:32:25.125507 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b529d68-11ad-4813-86b8-d3ec669d3230 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.125793 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b529d68-11ad-4813-86b8-d3ec669d3230 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.126070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b529d68-11ad-4813-86b8-d3ec669d3230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.167053 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-3b529d68-11ad-4813-86b8-d3ec669d3230 None None] MFA Rules not processed for user `ec50ac7c54324884a1b2a6c63d617907`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:25.168303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3b529d68-11ad-4813-86b8-d3ec669d3230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:25.241869 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1439/2874] 10.222.0.100 () {58 vars in 1023 bytes} [Mon Aug 1 15:32:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:25.253465 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24497cd3-3db1-457e-812f-38da49c6d049 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:25.264877 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24497cd3-3db1-457e-812f-38da49c6d049 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:25.268515 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ce205-9a56-4b2e-b6be-c74fa7013a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:25.269125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ce205-9a56-4b2e-b6be-c74fa7013a45 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:25.269408 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ce205-9a56-4b2e-b6be-c74fa7013a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:25.269682 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ce205-9a56-4b2e-b6be-c74fa7013a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:25.280162 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ce205-9a56-4b2e-b6be-c74fa7013a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:25.281590 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ce205-9a56-4b2e-b6be-c74fa7013a45 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:25.293162 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1436/2875] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:32:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:27.668393 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b529d68-11ad-4813-86b8-d3ec669d3230 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:27.669913 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-3b529d68-11ad-4813-86b8-d3ec669d3230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:27.685655 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b529d68-11ad-4813-86b8-d3ec669d3230 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:27.692028 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3baa6ee-5c7e-4d16-ae4b-e8ac49ded666 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:27.692663 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3baa6ee-5c7e-4d16-ae4b-e8ac49ded666 service nova] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:27.692947 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3baa6ee-5c7e-4d16-ae4b-e8ac49ded666 service nova] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:27.693254 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3baa6ee-5c7e-4d16-ae4b-e8ac49ded666 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:27.695106 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-f3baa6ee-5c7e-4d16-ae4b-e8ac49ded666 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:27.708466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3baa6ee-5c7e-4d16-ae4b-e8ac49ded666 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:27.710066 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3baa6ee-5c7e-4d16-ae4b-e8ac49ded666 service nova] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:27.721303 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1440/2876] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:32:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:35.288160 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=51 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:37.716663 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=88 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:44.258444 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ce205-9a56-4b2e-b6be-c74fa7013a45 service neutron] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:44.322206 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ce205-9a56-4b2e-b6be-c74fa7013a45 service neutron] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:44.327760 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd038574-94cd-4ae2-8a26-03e085a77e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:44.328420 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd038574-94cd-4ae2-8a26-03e085a77e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:44.328710 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd038574-94cd-4ae2-8a26-03e085a77e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:44.328984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd038574-94cd-4ae2-8a26-03e085a77e36 admin admin] PATH_INFO: `/v3/users/9040a1d570d54c9cbadfca9cb4f7710d` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:44.330373 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd038574-94cd-4ae2-8a26-03e085a77e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9040a1d570d54c9cbadfca9cb4f7710d)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:44.332361 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd038574-94cd-4ae2-8a26-03e085a77e36 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:44.394789 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cd038574-94cd-4ae2-8a26-03e085a77e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9040a1d570d54c9cbadfca9cb4f7710d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:44.405293 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cd038574-94cd-4ae2-8a26-03e085a77e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9040a1d570d54c9cbadfca9cb4f7710d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:44.406059 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cd038574-94cd-4ae2-8a26-03e085a77e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9040a1d570d54c9cbadfca9cb4f7710d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:44.407004 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cd038574-94cd-4ae2-8a26-03e085a77e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9040a1d570d54c9cbadfca9cb4f7710d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:44.424672 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-cd038574-94cd-4ae2-8a26-03e085a77e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9040a1d570d54c9cbadfca9cb4f7710d'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:44.439793 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1437/2877] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:32:44 2022] DELETE /identity/v3/users/9040a1d570d54c9cbadfca9cb4f7710d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:44.445654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf734c-7f27-4eb8-b630-2731182d880c None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:44.446232 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf734c-7f27-4eb8-b630-2731182d880c None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:44.446513 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf734c-7f27-4eb8-b630-2731182d880c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:44.469455 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-61bf734c-7f27-4eb8-b630-2731182d880c None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:44.478113 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-61bf734c-7f27-4eb8-b630-2731182d880c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:44.555905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1441/2878] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:44.565855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd038574-94cd-4ae2-8a26-03e085a77e36 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:44.567443 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-cd038574-94cd-4ae2-8a26-03e085a77e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:44.635036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd038574-94cd-4ae2-8a26-03e085a77e36 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:44.640039 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be936cc-90cb-48e7-ad1c-cffc596e6531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:44.640899 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be936cc-90cb-48e7-ad1c-cffc596e6531 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:44.641424 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be936cc-90cb-48e7-ad1c-cffc596e6531 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:44.641707 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be936cc-90cb-48e7-ad1c-cffc596e6531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:44.652304 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be936cc-90cb-48e7-ad1c-cffc596e6531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:44.653792 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be936cc-90cb-48e7-ad1c-cffc596e6531 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:44.665152 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1438/2879] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:32:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:45.119187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf734c-7f27-4eb8-b630-2731182d880c None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:45.120722 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-61bf734c-7f27-4eb8-b630-2731182d880c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:45.134588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf734c-7f27-4eb8-b630-2731182d880c None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:45.140316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480fca3-5b15-4829-a07d-900412fcb53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:45.140979 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480fca3-5b15-4829-a07d-900412fcb53f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:45.141258 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480fca3-5b15-4829-a07d-900412fcb53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:45.141533 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480fca3-5b15-4829-a07d-900412fcb53f admin admin] PATH_INFO: `/v3/projects/e3d9d9c6b7bb43e98a0a3865e44a9626` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:45.142770 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480fca3-5b15-4829-a07d-900412fcb53f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d9d9c6b7bb43e98a0a3865e44a9626)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:45.145001 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480fca3-5b15-4829-a07d-900412fcb53f admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:45.204080 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0480fca3-5b15-4829-a07d-900412fcb53f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d9d9c6b7bb43e98a0a3865e44a9626', 'request_id': 'req-0480fca3-5b15-4829-a07d-900412fcb53f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:45.211324 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0480fca3-5b15-4829-a07d-900412fcb53f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d9d9c6b7bb43e98a0a3865e44a9626', 'request_id': 'req-0480fca3-5b15-4829-a07d-900412fcb53f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:45.217566 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0480fca3-5b15-4829-a07d-900412fcb53f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d9d9c6b7bb43e98a0a3865e44a9626', 'request_id': 'req-0480fca3-5b15-4829-a07d-900412fcb53f'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:45.230136 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0480fca3-5b15-4829-a07d-900412fcb53f admin admin] The token cache is being invalidate because project e3d9d9c6b7bb43e98a0a3865e44a9626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:32:45.230432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0480fca3-5b15-4829-a07d-900412fcb53f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:45.231096 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-0480fca3-5b15-4829-a07d-900412fcb53f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:32:45.232911 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1442/2880] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:32:45 2022] DELETE /identity/v3/projects/e3d9d9c6b7bb43e98a0a3865e44a9626 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:45.237521 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bbd8e-5d0f-4ff6-b3e5-5d8cb4d31df8 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:45.237815 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bbd8e-5d0f-4ff6-b3e5-5d8cb4d31df8 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:45.238171 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bbd8e-5d0f-4ff6-b3e5-5d8cb4d31df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:45.258295 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-872bbd8e-5d0f-4ff6-b3e5-5d8cb4d31df8 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:45.259483 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-872bbd8e-5d0f-4ff6-b3e5-5d8cb4d31df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:45.278917 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:45.329738 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1439/2881] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:45.337815 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5d84c-7ffc-4447-b8d6-3eec711275fe None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:45.338098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5d84c-7ffc-4447-b8d6-3eec711275fe None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:45.338375 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5d84c-7ffc-4447-b8d6-3eec711275fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:45.358163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-bee5d84c-7ffc-4447-b8d6-3eec711275fe None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:45.359607 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-bee5d84c-7ffc-4447-b8d6-3eec711275fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:45.367981 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1443/2882] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:45.372348 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bbd8e-5d0f-4ff6-b3e5-5d8cb4d31df8 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:45.383207 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bbd8e-5d0f-4ff6-b3e5-5d8cb4d31df8 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:45.387948 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ea641-3c6a-406b-99ea-fc6f31e5f0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:45.388583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ea641-3c6a-406b-99ea-fc6f31e5f0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:45.388933 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ea641-3c6a-406b-99ea-fc6f31e5f0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:45.389703 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ea641-3c6a-406b-99ea-fc6f31e5f0b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:45.390413 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ea641-3c6a-406b-99ea-fc6f31e5f0b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:32:45.390749 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ea641-3c6a-406b-99ea-fc6f31e5f0b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:45.392384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645ea641-3c6a-406b-99ea-fc6f31e5f0b3 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:45.405370 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1440/2883] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:32:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:32:45.409799 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5d84c-7ffc-4447-b8d6-3eec711275fe None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:45.419539 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5d84c-7ffc-4447-b8d6-3eec711275fe None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:45.424872 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f385fb-eadf-41d4-85cd-f5dd5f3b8074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:45.425502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f385fb-eadf-41d4-85cd-f5dd5f3b8074 admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:45.425783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f385fb-eadf-41d4-85cd-f5dd5f3b8074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:45.426060 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f385fb-eadf-41d4-85cd-f5dd5f3b8074 admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:45.426783 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f385fb-eadf-41d4-85cd-f5dd5f3b8074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:45.428781 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f385fb-eadf-41d4-85cd-f5dd5f3b8074 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:45.459852 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1444/2884] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:32:45 2022] POST /identity/v3/projects => generated 370 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:45.464421 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ea641-3c6a-406b-99ea-fc6f31e5f0b3 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:45.475258 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ea641-3c6a-406b-99ea-fc6f31e5f0b3 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:45.533816 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea2fe0-6961-48e2-afdd-24c12c96c7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:45.534343 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea2fe0-6961-48e2-afdd-24c12c96c7a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:45.534626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea2fe0-6961-48e2-afdd-24c12c96c7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:45.534935 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea2fe0-6961-48e2-afdd-24c12c96c7a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:45.535626 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea2fe0-6961-48e2-afdd-24c12c96c7a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:45.537590 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea2fe0-6961-48e2-afdd-24c12c96c7a8 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:45.560377 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1441/2885] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:32:45 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:32:45.564760 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f385fb-eadf-41d4-85cd-f5dd5f3b8074 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:45.575997 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f385fb-eadf-41d4-85cd-f5dd5f3b8074 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:45.581340 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632058b-a1d7-4459-8ca7-3dc6376d46d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:45.581968 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632058b-a1d7-4459-8ca7-3dc6376d46d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:45.582251 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632058b-a1d7-4459-8ca7-3dc6376d46d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:45.582525 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632058b-a1d7-4459-8ca7-3dc6376d46d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:45.583275 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632058b-a1d7-4459-8ca7-3dc6376d46d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:45.584882 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632058b-a1d7-4459-8ca7-3dc6376d46d2 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:45.596070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1445/2886] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:32:45 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:45.600202 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea2fe0-6961-48e2-afdd-24c12c96c7a8 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:45.611492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea2fe0-6961-48e2-afdd-24c12c96c7a8 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:45.621940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ce888-7bc4-410c-b26b-43c925dacf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:45.621940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce888-7bc4-410c-b26b-43c925dacf70 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:45.621940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce888-7bc4-410c-b26b-43c925dacf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:45.621940 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ce888-7bc4-410c-b26b-43c925dacf70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:45.622462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ce888-7bc4-410c-b26b-43c925dacf70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:45.622462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ce888-7bc4-410c-b26b-43c925dacf70 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:45.632560 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1442/2887] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:32:45 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:32:45.636837 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632058b-a1d7-4459-8ca7-3dc6376d46d2 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:45.650535 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632058b-a1d7-4459-8ca7-3dc6376d46d2 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:45.655395 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807e29f-0931-4ba5-a0ee-522858882dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:45.656069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807e29f-0931-4ba5-a0ee-522858882dca admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:45.656350 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807e29f-0931-4ba5-a0ee-522858882dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:45.656631 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807e29f-0931-4ba5-a0ee-522858882dca admin admin] PATH_INFO: `/v3/projects/a2cc365b6ffc4cde97c7ecef279d559b/users/c0f45cf72a774146a15c22118554dd74/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:45.673466 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807e29f-0931-4ba5-a0ee-522858882dca admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2cc365b6ffc4cde97c7ecef279d559b, user_id=c0f45cf72a774146a15c22118554dd74, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:45.676179 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807e29f-0931-4ba5-a0ee-522858882dca admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:45.692262 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1446/2888] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:32:45 2022] PUT /identity/v3/projects/a2cc365b6ffc4cde97c7ecef279d559b/users/c0f45cf72a774146a15c22118554dd74/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:32:45.702941 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c38a8-8a25-47fa-9c8e-25e4bb4a1bce None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:45.703284 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c38a8-8a25-47fa-9c8e-25e4bb4a1bce None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:45.703566 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c38a8-8a25-47fa-9c8e-25e4bb4a1bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:45.734979 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-ec6c38a8-8a25-47fa-9c8e-25e4bb4a1bce None None] MFA Rules not processed for user `c0f45cf72a774146a15c22118554dd74`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:32:45.736235 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ec6c38a8-8a25-47fa-9c8e-25e4bb4a1bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:45.818115 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1443/2889] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:32:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:32:45.836745 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807e29f-0931-4ba5-a0ee-522858882dca admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:32:45.838802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-9807e29f-0931-4ba5-a0ee-522858882dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:32:45.868116 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:45.932499 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807e29f-0931-4ba5-a0ee-522858882dca admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:32:45.938236 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097a100-63e2-4df5-b5c0-b51b0ca46cbb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e75edfc5bb404d63b27a355d5ab8af01', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:32:45.939274 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097a100-63e2-4df5-b5c0-b51b0ca46cbb service glance] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:32:45.939724 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097a100-63e2-4df5-b5c0-b51b0ca46cbb service glance] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:32:45.940139 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097a100-63e2-4df5-b5c0-b51b0ca46cbb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:32:45.951071 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097a100-63e2-4df5-b5c0-b51b0ca46cbb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:32:45.952682 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097a100-63e2-4df5-b5c0-b51b0ca46cbb service glance] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:32:45.963720 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1447/2890] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:32:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:32:55.814915 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:32:55.959524 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=26 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:21.428220 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c38a8-8a25-47fa-9c8e-25e4bb4a1bce None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:21.510673 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c38a8-8a25-47fa-9c8e-25e4bb4a1bce None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:21.519500 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952576d-62c3-4ca1-8070-b5c5eb8c6587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:21.520155 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952576d-62c3-4ca1-8070-b5c5eb8c6587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:21.520456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952576d-62c3-4ca1-8070-b5c5eb8c6587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:21.520735 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952576d-62c3-4ca1-8070-b5c5eb8c6587 admin admin] PATH_INFO: `/v3/users/c0f45cf72a774146a15c22118554dd74` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:21.521953 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952576d-62c3-4ca1-8070-b5c5eb8c6587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0f45cf72a774146a15c22118554dd74)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:21.523961 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952576d-62c3-4ca1-8070-b5c5eb8c6587 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:21.573319 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1952576d-62c3-4ca1-8070-b5c5eb8c6587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0f45cf72a774146a15c22118554dd74'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:21.581510 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1952576d-62c3-4ca1-8070-b5c5eb8c6587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0f45cf72a774146a15c22118554dd74'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:21.582140 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1952576d-62c3-4ca1-8070-b5c5eb8c6587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0f45cf72a774146a15c22118554dd74'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:21.582835 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1952576d-62c3-4ca1-8070-b5c5eb8c6587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0f45cf72a774146a15c22118554dd74'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:21.595799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-1952576d-62c3-4ca1-8070-b5c5eb8c6587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0f45cf72a774146a15c22118554dd74'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:21.607823 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1444/2891] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:33:21 2022] DELETE /identity/v3/users/c0f45cf72a774146a15c22118554dd74 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:21.612328 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8d720-62f6-40c8-93d8-dc0ad21ac9c3 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:21.612832 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8d720-62f6-40c8-93d8-dc0ad21ac9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:21.612832 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8d720-62f6-40c8-93d8-dc0ad21ac9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:21.651441 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-64c8d720-62f6-40c8-93d8-dc0ad21ac9c3 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:21.652632 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-64c8d720-62f6-40c8-93d8-dc0ad21ac9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:21.710958 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1448/2892] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:21.721067 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952576d-62c3-4ca1-8070-b5c5eb8c6587 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:21.722527 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1952576d-62c3-4ca1-8070-b5c5eb8c6587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:21.781680 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952576d-62c3-4ca1-8070-b5c5eb8c6587 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:21.785492 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8edf6-7276-45b0-b524-a5344c23bbe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:21.786182 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8edf6-7276-45b0-b524-a5344c23bbe3 service neutron] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:21.786466 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8edf6-7276-45b0-b524-a5344c23bbe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:21.786781 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8edf6-7276-45b0-b524-a5344c23bbe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:21.795907 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8edf6-7276-45b0-b524-a5344c23bbe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:21.797303 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8edf6-7276-45b0-b524-a5344c23bbe3 service neutron] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:21.806799 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1445/2893] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:33:22.070586 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8d720-62f6-40c8-93d8-dc0ad21ac9c3 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:22.072070 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-64c8d720-62f6-40c8-93d8-dc0ad21ac9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:22.085098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8d720-62f6-40c8-93d8-dc0ad21ac9c3 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:22.090243 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabbda3-3c12-4a4c-b79a-bd153987c738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:22.090932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabbda3-3c12-4a4c-b79a-bd153987c738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:22.091240 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabbda3-3c12-4a4c-b79a-bd153987c738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:22.091551 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabbda3-3c12-4a4c-b79a-bd153987c738 admin admin] PATH_INFO: `/v3/projects/a2cc365b6ffc4cde97c7ecef279d559b` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:22.093525 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabbda3-3c12-4a4c-b79a-bd153987c738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2cc365b6ffc4cde97c7ecef279d559b)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:22.096000 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabbda3-3c12-4a4c-b79a-bd153987c738 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:22.144494 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2cabbda3-3c12-4a4c-b79a-bd153987c738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2cc365b6ffc4cde97c7ecef279d559b', 'request_id': 'req-2cabbda3-3c12-4a4c-b79a-bd153987c738'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:22.150523 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2cabbda3-3c12-4a4c-b79a-bd153987c738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2cc365b6ffc4cde97c7ecef279d559b', 'request_id': 'req-2cabbda3-3c12-4a4c-b79a-bd153987c738'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:22.156712 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2cabbda3-3c12-4a4c-b79a-bd153987c738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2cc365b6ffc4cde97c7ecef279d559b', 'request_id': 'req-2cabbda3-3c12-4a4c-b79a-bd153987c738'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:22.169483 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2cabbda3-3c12-4a4c-b79a-bd153987c738 admin admin] The token cache is being invalidate because project a2cc365b6ffc4cde97c7ecef279d559b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:33:22.169787 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2cabbda3-3c12-4a4c-b79a-bd153987c738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:22.170549 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-2cabbda3-3c12-4a4c-b79a-bd153987c738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:33:22.171995 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1449/2894] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:33:22 2022] DELETE /identity/v3/projects/a2cc365b6ffc4cde97c7ecef279d559b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:22.178862 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844a02c-9f13-4df9-a8fd-2beed5717ea5 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:22.179181 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844a02c-9f13-4df9-a8fd-2beed5717ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:22.179460 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844a02c-9f13-4df9-a8fd-2beed5717ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:22.185981 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:22.222570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-1844a02c-9f13-4df9-a8fd-2beed5717ea5 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:22.223776 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-1844a02c-9f13-4df9-a8fd-2beed5717ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:22.284845 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1446/2895] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:22.295562 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a73b73-fad5-4dd4-99dd-1dddf9180292 None None] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:22.295850 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a73b73-fad5-4dd4-99dd-1dddf9180292 None None] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:22.296132 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a73b73-fad5-4dd4-99dd-1dddf9180292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:22.316268 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.auth.core [None req-89a73b73-fad5-4dd4-99dd-1dddf9180292 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80358) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:22.317491 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-89a73b73-fad5-4dd4-99dd-1dddf9180292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:22.326279 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1450/2896] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:22.331315 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844a02c-9f13-4df9-a8fd-2beed5717ea5 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:22.341063 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844a02c-9f13-4df9-a8fd-2beed5717ea5 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:22.346473 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c4b1c-ba90-4f1c-a297-a876a00202a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:22.347119 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c4b1c-ba90-4f1c-a297-a876a00202a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:22.347401 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c4b1c-ba90-4f1c-a297-a876a00202a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:22.347675 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c4b1c-ba90-4f1c-a297-a876a00202a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:22.348371 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c4b1c-ba90-4f1c-a297-a876a00202a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80359) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 15:33:22.348677 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c4b1c-ba90-4f1c-a297-a876a00202a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:22.350283 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c4b1c-ba90-4f1c-a297-a876a00202a2 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:22.362025 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1447/2897] 10.222.0.100 () {58 vars in 1158 bytes} [Mon Aug 1 15:33:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 15:33:22.367420 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a73b73-fad5-4dd4-99dd-1dddf9180292 None None] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:22.377196 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a73b73-fad5-4dd4-99dd-1dddf9180292 None None] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:22.382373 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d379c7f-dd96-40ee-93ab-5843647acb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:22.383042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d379c7f-dd96-40ee-93ab-5843647acb1b admin admin] REQUEST_METHOD: `POST` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:22.383326 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d379c7f-dd96-40ee-93ab-5843647acb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:22.383603 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d379c7f-dd96-40ee-93ab-5843647acb1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:22.384287 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d379c7f-dd96-40ee-93ab-5843647acb1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:22.386237 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d379c7f-dd96-40ee-93ab-5843647acb1b admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:22.428448 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1451/2898] 10.222.0.100 () {60 vars in 1158 bytes} [Mon Aug 1 15:33:22 2022] POST /identity/v3/projects => generated 364 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:22.432559 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c4b1c-ba90-4f1c-a297-a876a00202a2 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:22.443591 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c4b1c-ba90-4f1c-a297-a876a00202a2 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:22.504305 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a49f56-f551-47e0-b7bc-fb49feb264e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:22.504870 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a49f56-f551-47e0-b7bc-fb49feb264e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:22.505157 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a49f56-f551-47e0-b7bc-fb49feb264e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:22.505449 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a49f56-f551-47e0-b7bc-fb49feb264e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:22.506169 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a49f56-f551-47e0-b7bc-fb49feb264e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:22.508139 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a49f56-f551-47e0-b7bc-fb49feb264e4 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:22.529409 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1448/2899] 10.222.0.100 () {60 vars in 1149 bytes} [Mon Aug 1 15:33:22 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 15:33:22.534285 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d379c7f-dd96-40ee-93ab-5843647acb1b admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:22.545173 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d379c7f-dd96-40ee-93ab-5843647acb1b admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:22.552174 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12106522-4fad-42ae-987b-47ae8a44e1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:22.552811 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12106522-4fad-42ae-987b-47ae8a44e1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:22.553120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12106522-4fad-42ae-987b-47ae8a44e1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:22.553368 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12106522-4fad-42ae-987b-47ae8a44e1b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:22.554069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12106522-4fad-42ae-987b-47ae8a44e1b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:22.555785 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12106522-4fad-42ae-987b-47ae8a44e1b4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:22.568751 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1452/2900] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:33:22 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:22.572866 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a49f56-f551-47e0-b7bc-fb49feb264e4 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:22.583384 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a49f56-f551-47e0-b7bc-fb49feb264e4 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:22.588241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac77d4c-611e-421a-9b17-a454c058d46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:22.588858 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac77d4c-611e-421a-9b17-a454c058d46f admin admin] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:22.589143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac77d4c-611e-421a-9b17-a454c058d46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:22.589456 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac77d4c-611e-421a-9b17-a454c058d46f admin admin] PATH_INFO: `/v3/roles` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:22.590154 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac77d4c-611e-421a-9b17-a454c058d46f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:22.591857 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac77d4c-611e-421a-9b17-a454c058d46f admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:22.604570 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1449/2901] 10.222.0.100 () {58 vars in 1127 bytes} [Mon Aug 1 15:33:22 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 15:33:22.609849 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12106522-4fad-42ae-987b-47ae8a44e1b4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:22.620603 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12106522-4fad-42ae-987b-47ae8a44e1b4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:22.625885 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeea8d7-d1c5-4edb-b964-a5e23b85c622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:22.626556 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeea8d7-d1c5-4edb-b964-a5e23b85c622 admin admin] REQUEST_METHOD: `PUT` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:22.626884 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeea8d7-d1c5-4edb-b964-a5e23b85c622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:22.627166 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeea8d7-d1c5-4edb-b964-a5e23b85c622 admin admin] PATH_INFO: `/v3/projects/a5861db186554b4ebfae39453e4307af/users/e4ddb434af6548ac83bedad07de88796/roles/d1f6973ce11a4644b649b38f8b2df173` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:22.642385 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeea8d7-d1c5-4edb-b964-a5e23b85c622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5861db186554b4ebfae39453e4307af, user_id=e4ddb434af6548ac83bedad07de88796, role_id=d1f6973ce11a4644b649b38f8b2df173)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:22.645100 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeea8d7-d1c5-4edb-b964-a5e23b85c622 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:22.647308 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:22.659535 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1453/2902] 10.222.0.100 () {60 vars in 1488 bytes} [Mon Aug 1 15:33:22 2022] PUT /identity/v3/projects/a5861db186554b4ebfae39453e4307af/users/e4ddb434af6548ac83bedad07de88796/roles/d1f6973ce11a4644b649b38f8b2df173 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:33:22.669930 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bd148-1d91-4c0a-95d7-17b2398a65ae None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:22.670309 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bd148-1d91-4c0a-95d7-17b2398a65ae None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:22.670686 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bd148-1d91-4c0a-95d7-17b2398a65ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:22.699984 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-128bd148-1d91-4c0a-95d7-17b2398a65ae None None] MFA Rules not processed for user `e4ddb434af6548ac83bedad07de88796`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:33:22.701224 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-128bd148-1d91-4c0a-95d7-17b2398a65ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:22.746880 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:22.770732 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1450/2903] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:33:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:33:22.782318 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeea8d7-d1c5-4edb-b964-a5e23b85c622 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:22.784588 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4aeea8d7-d1c5-4edb-b964-a5e23b85c622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:22.847651 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeea8d7-d1c5-4edb-b964-a5e23b85c622 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:22.851674 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1c9f4-400e-4cb9-9cfd-ae732c50f8e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead1b36d35f3485f89edd15b8443feea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:22.852413 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1c9f4-400e-4cb9-9cfd-ae732c50f8e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:22.852693 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1c9f4-400e-4cb9-9cfd-ae732c50f8e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:22.852965 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1c9f4-400e-4cb9-9cfd-ae732c50f8e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:22.862471 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1c9f4-400e-4cb9-9cfd-ae732c50f8e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:22.863948 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1c9f4-400e-4cb9-9cfd-ae732c50f8e0 service cinder] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:22.872851 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1454/2904] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:33:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:32.767583 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=7 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:32.868753 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:33:40.614255 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bd148-1d91-4c0a-95d7-17b2398a65ae None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:33:40.615746 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-128bd148-1d91-4c0a-95d7-17b2398a65ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:40.709844 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bd148-1d91-4c0a-95d7-17b2398a65ae None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:33:40.715197 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972ae609-b3a6-460d-8ae6-3e1d156db8fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:33:40.715814 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ae609-b3a6-460d-8ae6-3e1d156db8fc service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:33:40.716097 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ae609-b3a6-460d-8ae6-3e1d156db8fc service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:33:40.716374 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972ae609-b3a6-460d-8ae6-3e1d156db8fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:33:40.718177 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-972ae609-b3a6-460d-8ae6-3e1d156db8fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:33:40.729862 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ae609-b3a6-460d-8ae6-3e1d156db8fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:33:40.731512 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972ae609-b3a6-460d-8ae6-3e1d156db8fc service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:33:40.742125 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1451/2905] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:33:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:33:50.737578 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=41 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:05.484534 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1c9f4-400e-4cb9-9cfd-ae732c50f8e0 service cinder] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:05.574224 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1c9f4-400e-4cb9-9cfd-ae732c50f8e0 service cinder] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:05.579695 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcef29-2a6f-4073-a3c7-4a68fc2f9bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:05.580342 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcef29-2a6f-4073-a3c7-4a68fc2f9bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:05.580624 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcef29-2a6f-4073-a3c7-4a68fc2f9bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:05.580903 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcef29-2a6f-4073-a3c7-4a68fc2f9bd4 admin admin] PATH_INFO: `/v3/users/e4ddb434af6548ac83bedad07de88796` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:05.582069 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcef29-2a6f-4073-a3c7-4a68fc2f9bd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4ddb434af6548ac83bedad07de88796)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:05.584036 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcef29-2a6f-4073-a3c7-4a68fc2f9bd4 admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:05.637438 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-34fcef29-2a6f-4073-a3c7-4a68fc2f9bd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4ddb434af6548ac83bedad07de88796'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:05.638120 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-34fcef29-2a6f-4073-a3c7-4a68fc2f9bd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4ddb434af6548ac83bedad07de88796'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:05.645119 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-34fcef29-2a6f-4073-a3c7-4a68fc2f9bd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4ddb434af6548ac83bedad07de88796'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:05.645797 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-34fcef29-2a6f-4073-a3c7-4a68fc2f9bd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4ddb434af6548ac83bedad07de88796'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:05.657998 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-34fcef29-2a6f-4073-a3c7-4a68fc2f9bd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4ddb434af6548ac83bedad07de88796'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:05.674088 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1455/2906] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:34:05 2022] DELETE /identity/v3/users/e4ddb434af6548ac83bedad07de88796 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:05.678701 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d056f7-4b1c-4e0d-b87d-7dae939e1876 None None] REQUEST_METHOD: `POST` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:05.679229 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d056f7-4b1c-4e0d-b87d-7dae939e1876 None None] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:05.679346 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d056f7-4b1c-4e0d-b87d-7dae939e1876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:05.699665 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.auth.core [None req-29d056f7-4b1c-4e0d-b87d-7dae939e1876 None None] MFA Rules not processed for user `24f06d52227a4059a53df8c2c849eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80359) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 15:34:05.700925 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-29d056f7-4b1c-4e0d-b87d-7dae939e1876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:05.762435 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1452/2907] 10.222.0.100 () {58 vars in 963 bytes} [Mon Aug 1 15:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 15:34:05.774273 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcef29-2a6f-4073-a3c7-4a68fc2f9bd4 admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:05.775895 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-34fcef29-2a6f-4073-a3c7-4a68fc2f9bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:05.839127 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcef29-2a6f-4073-a3c7-4a68fc2f9bd4 admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:05.843543 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b7a4a-219d-4966-a653-ab398457a4fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:05.844160 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b7a4a-219d-4966-a653-ab398457a4fd service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:05.844435 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b7a4a-219d-4966-a653-ab398457a4fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:05.844705 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b7a4a-219d-4966-a653-ab398457a4fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:05.854432 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b7a4a-219d-4966-a653-ab398457a4fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:05.855910 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b7a4a-219d-4966-a653-ab398457a4fd service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:05.866281 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1456/2908] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:34:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:06.162210 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d056f7-4b1c-4e0d-b87d-7dae939e1876 None None] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:06.164171 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-29d056f7-4b1c-4e0d-b87d-7dae939e1876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:06.186010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d056f7-4b1c-4e0d-b87d-7dae939e1876 None None] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:06.192389 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:06.193160 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:06.193516 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:06.193855 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] PATH_INFO: `/v3/projects/a5861db186554b4ebfae39453e4307af` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:06.195223 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5861db186554b4ebfae39453e4307af)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:06.197345 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:06.252036 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5861db186554b4ebfae39453e4307af', 'request_id': 'req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:06.259364 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5861db186554b4ebfae39453e4307af', 'request_id': 'req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:06.268790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5861db186554b4ebfae39453e4307af', 'request_id': 'req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:06.280871 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] The token cache is being invalidate because project a5861db186554b4ebfae39453e4307af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80359) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:06.281143 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:06.281788 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:06.283352 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1453/2909] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:34:06 2022] DELETE /identity/v3/projects/a5861db186554b4ebfae39453e4307af => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:15.861782 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:16.277104 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:40.399056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b7a4a-219d-4966-a653-ab398457a4fd service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:40.408683 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b7a4a-219d-4966-a653-ab398457a4fd service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:40.459806 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d5991-3994-4609-958a-c7aef492be20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:40.460433 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d5991-3994-4609-958a-c7aef492be20 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:40.460718 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d5991-3994-4609-958a-c7aef492be20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:40.460993 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d5991-3994-4609-958a-c7aef492be20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:40.462861 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-750d5991-3994-4609-958a-c7aef492be20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:40.528711 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d5991-3994-4609-958a-c7aef492be20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:40.530574 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d5991-3994-4609-958a-c7aef492be20 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:40.545919 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1457/2910] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:34:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:41.454479 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:41.456055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:41.624850 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8da5a0-a847-40f1-80db-cb6174fd8ab5 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:41.630878 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e98ed0-4775-43d5-92a1-cd082e35135a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec50ac7c54324884a1b2a6c63d617907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:41.631605 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e98ed0-4775-43d5-92a1-cd082e35135a service nova] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:41.631958 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e98ed0-4775-43d5-92a1-cd082e35135a service nova] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:41.632306 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e98ed0-4775-43d5-92a1-cd082e35135a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:41.634531 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-55e98ed0-4775-43d5-92a1-cd082e35135a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:41.649689 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e98ed0-4775-43d5-92a1-cd082e35135a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:41.651528 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e98ed0-4775-43d5-92a1-cd082e35135a service nova] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:41.664860 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1454/2911] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:34:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 15:34:45.396098 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d5991-3994-4609-958a-c7aef492be20 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:45.397563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-750d5991-3994-4609-958a-c7aef492be20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:45.409316 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d5991-3994-4609-958a-c7aef492be20 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:45.412969 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d254d-a003-4687-9e5f-0e6396394ac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f757eabcb84de6ab042e0ac5a8158c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:45.413596 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d254d-a003-4687-9e5f-0e6396394ac1 service neutron] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:45.413880 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d254d-a003-4687-9e5f-0e6396394ac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:45.414163 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d254d-a003-4687-9e5f-0e6396394ac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:45.416002 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-339d254d-a003-4687-9e5f-0e6396394ac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:45.482187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d254d-a003-4687-9e5f-0e6396394ac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:45.483650 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d254d-a003-4687-9e5f-0e6396394ac1 service neutron] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:45.493502 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1458/2912] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:48.841142 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e98ed0-4775-43d5-92a1-cd082e35135a service nova] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:48.842640 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-55e98ed0-4775-43d5-92a1-cd082e35135a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:48.957763 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e98ed0-4775-43d5-92a1-cd082e35135a service nova] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:48.963200 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7109b56-2eed-4651-b35e-06287e37c65b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:48.963874 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7109b56-2eed-4651-b35e-06287e37c65b service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:48.964156 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7109b56-2eed-4651-b35e-06287e37c65b service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:48.964439 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7109b56-2eed-4651-b35e-06287e37c65b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:48.966552 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c7109b56-2eed-4651-b35e-06287e37c65b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:48.979587 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7109b56-2eed-4651-b35e-06287e37c65b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:48.981208 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7109b56-2eed-4651-b35e-06287e37c65b service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:48.993936 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1455/2913] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:34:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:52.277508 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d254d-a003-4687-9e5f-0e6396394ac1 service neutron] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:52.298200 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d254d-a003-4687-9e5f-0e6396394ac1 service neutron] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:52.307823 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0c634-befd-432c-8c56-8baa692db418 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:52.309053 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0c634-befd-432c-8c56-8baa692db418 service placement] REQUEST_METHOD: `GET` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:52.309597 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0c634-befd-432c-8c56-8baa692db418 service placement] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:52.310175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e0c634-befd-432c-8c56-8baa692db418 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:52.318553 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-b3e0c634-befd-432c-8c56-8baa692db418 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:52.335429 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0c634-befd-432c-8c56-8baa692db418 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:52.337498 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e0c634-befd-432c-8c56-8baa692db418 service placement] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:52.350099 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1459/2914] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:34:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:34:52.364691 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7109b56-2eed-4651-b35e-06287e37c65b service placement] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:52.366269 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-c7109b56-2eed-4651-b35e-06287e37c65b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:52.385742 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7109b56-2eed-4651-b35e-06287e37c65b service placement] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:52.396417 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f0e95-f292-40ac-be77-deb830820036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:52.397091 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f0e95-f292-40ac-be77-deb830820036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:52.397375 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f0e95-f292-40ac-be77-deb830820036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:52.397654 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f0e95-f292-40ac-be77-deb830820036 admin admin] PATH_INFO: `/v3/users/250e40933543431981bcb19db78fa144` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:52.399055 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f0e95-f292-40ac-be77-deb830820036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=250e40933543431981bcb19db78fa144)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:52.401010 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f0e95-f292-40ac-be77-deb830820036 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:52.449851 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:52.453491 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-902f0e95-f292-40ac-be77-deb830820036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '250e40933543431981bcb19db78fa144'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.462540 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-902f0e95-f292-40ac-be77-deb830820036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '250e40933543431981bcb19db78fa144'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.463462 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-902f0e95-f292-40ac-be77-deb830820036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '250e40933543431981bcb19db78fa144'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.464211 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-902f0e95-f292-40ac-be77-deb830820036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '250e40933543431981bcb19db78fa144'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.480486 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-902f0e95-f292-40ac-be77-deb830820036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '250e40933543431981bcb19db78fa144'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.501997 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1456/2915] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:34:52 2022] DELETE /identity/v3/users/250e40933543431981bcb19db78fa144 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 15:34:52.712654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0c634-befd-432c-8c56-8baa692db418 service placement] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:52.731654 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e0c634-befd-432c-8c56-8baa692db418 service placement] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:52.768631 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=100 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:52.794563 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:52.795348 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:52.795640 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:52.795920 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] PATH_INFO: `/v3/projects/7cf605507bae49a0a797b96db9da31bd` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:52.797187 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cf605507bae49a0a797b96db9da31bd)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:52.799175 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:52.852042 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cf605507bae49a0a797b96db9da31bd', 'request_id': 'req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.862182 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cf605507bae49a0a797b96db9da31bd', 'request_id': 'req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.871095 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cf605507bae49a0a797b96db9da31bd', 'request_id': 'req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.890932 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] The token cache is being invalidate because project 7cf605507bae49a0a797b96db9da31bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:52.891423 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.892402 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:52.894838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1460/2916] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:34:52 2022] DELETE /identity/v3/projects/7cf605507bae49a0a797b96db9da31bd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:52.900161 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f0e95-f292-40ac-be77-deb830820036 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:52.901721 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-902f0e95-f292-40ac-be77-deb830820036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:52.916573 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f0e95-f292-40ac-be77-deb830820036 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:52.978203 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a5cee-b65c-437f-ac8d-e82fea7252f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:52.978783 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a5cee-b65c-437f-ac8d-e82fea7252f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:52.979065 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a5cee-b65c-437f-ac8d-e82fea7252f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:52.979339 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a5cee-b65c-437f-ac8d-e82fea7252f0 admin admin] PATH_INFO: `/v3/users/eb7f209a3cdc4500bc1381ba4ae12946` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:52.980558 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a5cee-b65c-437f-ac8d-e82fea7252f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb7f209a3cdc4500bc1381ba4ae12946)` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:52.982551 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a5cee-b65c-437f-ac8d-e82fea7252f0 admin admin] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:53.034503 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ef7a5cee-b65c-437f-ac8d-e82fea7252f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb7f209a3cdc4500bc1381ba4ae12946'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.042093 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ef7a5cee-b65c-437f-ac8d-e82fea7252f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb7f209a3cdc4500bc1381ba4ae12946'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.042891 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ef7a5cee-b65c-437f-ac8d-e82fea7252f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb7f209a3cdc4500bc1381ba4ae12946'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.043497 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ef7a5cee-b65c-437f-ac8d-e82fea7252f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb7f209a3cdc4500bc1381ba4ae12946'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.055853 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.notifications [None req-ef7a5cee-b65c-437f-ac8d-e82fea7252f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb7f209a3cdc4500bc1381ba4ae12946'} {{(pid=80359) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.069111 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1457/2917] 10.222.0.100 () {58 vars in 1229 bytes} [Mon Aug 1 15:34:52 2022] DELETE /identity/v3/users/eb7f209a3cdc4500bc1381ba4ae12946 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:34:53.264154 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] Authenticating user token {{(pid=80358) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:34:53.265838 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.fernet_utils [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80358) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:34:53.280919 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a0c98-5683-4224-a8d4-0730cc3df4ac admin admin] Validating token access rules against request {{(pid=80358) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:34:53.344824 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce155496-6e54-481b-b69a-662892eb366c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24f06d52227a4059a53df8c2c849eefc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bcc1ff25500841a3a7180e5fff03ff9f', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80358) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:34:53.345478 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce155496-6e54-481b-b69a-662892eb366c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:34:53.345767 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce155496-6e54-481b-b69a-662892eb366c admin admin] SCRIPT_NAME: `/identity` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:34:53.346039 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce155496-6e54-481b-b69a-662892eb366c admin admin] PATH_INFO: `/v3/projects/f69c93b17f034b04ad85b9c624890db8` {{(pid=80358) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:34:53.347297 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce155496-6e54-481b-b69a-662892eb366c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f69c93b17f034b04ad85b9c624890db8)` {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:34:53.349227 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce155496-6e54-481b-b69a-662892eb366c admin admin] RBAC: Authorization granted {{(pid=80358) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:34:53.409059 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce155496-6e54-481b-b69a-662892eb366c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f69c93b17f034b04ad85b9c624890db8', 'request_id': 'req-ce155496-6e54-481b-b69a-662892eb366c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.415802 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce155496-6e54-481b-b69a-662892eb366c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f69c93b17f034b04ad85b9c624890db8', 'request_id': 'req-ce155496-6e54-481b-b69a-662892eb366c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.422388 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce155496-6e54-481b-b69a-662892eb366c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f69c93b17f034b04ad85b9c624890db8', 'request_id': 'req-ce155496-6e54-481b-b69a-662892eb366c'} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.426056 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:34:53.438552 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce155496-6e54-481b-b69a-662892eb366c admin admin] The token cache is being invalidate because project f69c93b17f034b04ad85b9c624890db8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80358) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 15:34:53.438905 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce155496-6e54-481b-b69a-662892eb366c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.439567 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG keystone.notifications [None req-ce155496-6e54-481b-b69a-662892eb366c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80358) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 15:34:53.441115 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: [pid: 80358|app: 0|req: 1461/2918] 10.222.0.100 () {58 vars in 1238 bytes} [Mon Aug 1 15:34:53 2022] DELETE /identity/v3/projects/f69c93b17f034b04ad85b9c624890db8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 15:35:03.065533 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:35:03.437752 dv-cinder-843277-7-iscsi devstack@keystone.service[80358]: DEBUG dbcounter [-] [80358] Writing DB stats keystone:SELECT=2 {{(pid=80358) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 15:38:46.542226 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a5cee-b65c-437f-ac8d-e82fea7252f0 admin admin] Authenticating user token {{(pid=80359) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 15:38:46.543699 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-ef7a5cee-b65c-437f-ac8d-e82fea7252f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:38:46.613885 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a5cee-b65c-437f-ac8d-e82fea7252f0 admin admin] Validating token access rules against request {{(pid=80359) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 15:38:46.673790 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195022f5-8a5f-4d00-939e-7386f86e4fe0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bac5863e7bc4c7e854bd60912adb561', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6e147b6607b481591ceb250f7208c16', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80359) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 15:38:46.674412 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195022f5-8a5f-4d00-939e-7386f86e4fe0 service placement] REQUEST_METHOD: `GET` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 15:38:46.674716 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195022f5-8a5f-4d00-939e-7386f86e4fe0 service placement] SCRIPT_NAME: `/identity` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 15:38:46.674998 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195022f5-8a5f-4d00-939e-7386f86e4fe0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80359) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 15:38:46.676762 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.fernet_utils [None req-195022f5-8a5f-4d00-939e-7386f86e4fe0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80359) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 15:38:46.687968 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195022f5-8a5f-4d00-939e-7386f86e4fe0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 15:38:46.689541 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195022f5-8a5f-4d00-939e-7386f86e4fe0 service placement] RBAC: Authorization granted {{(pid=80359) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 15:38:46.700241 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: [pid: 80359|app: 0|req: 1458/2919] 10.222.0.100 () {60 vars in 1370 bytes} [Mon Aug 1 15:38:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 15:38:56.695216 dv-cinder-843277-7-iscsi devstack@keystone.service[80359]: DEBUG dbcounter [-] [80359] Writing DB stats keystone:SELECT=58 {{(pid=80359) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}